Eliciting and Aggregating Truthful and Noisy Information.
详细信息
文摘
In this dissertation,I study three information elicitation and aggregation methods,prediction markets,peer prediction mechanisms,and adaptive polling,using both theoretical and applied approaches. These methods mainly differ by their assumptions on the participants behavior,namely whether the participants possess noisy or perfect information and whether they strategically decide on what information to reveal. The first two methods,prediction markets and peer prediction mechanisms,assume that the participants are strategic and have perfect information. Their primary goal is to use carefully designed monetary rewards to incentivize the participants to truthfully reveal their private information. As a result,my studies of these methods focus on understanding to what extent are these methods incentive compatible in theory and in practice. The last method,adaptive polling,assumes that the participants are not strategic and have noisy information. I make four main contributions in this dissertation. First,I theoretically analyze how the participants knowledge of one anothers private information affects their strategic behavior when trading in a prediction market with a finite number of participants. When the participants private information is unconditionally independent,we show that the participants reveal their information as late as possible at any equilibrium,which is arguably the worse outcome for the purpose of information aggregation. We also provide insights on the equilibria of such prediction markets when the participants private information is both conditionally and unconditionally dependent given the outcome of the event. Second,I theoretically analyze the participants strategic behavior in a prediction market when a participant has outside incentives to manipulate the market probability. The presence of such outside incentives would seem to damage the information aggregation in the market. Surprisingly,when the existence of such incentives is certain and common knowledge,we show that there exist separating equilibria where all the participants private information is revealed and fully aggregated into the market probability. Third,I experimentally investigate participants behavior towards the peer prediction mechanisms,which were proposed to elicit information without observable ground truth. We conduct the first controlled online experiment of the Jurca and Faltings peer prediction mechanism,engaging the participants in a multiplayer,real-time and repeated game. Using a hidden Markov model to capture players strategies from their actions,our results show that participants successfully coordinate on uninformative equilibria and the truthful equilibrium is not focal,even when some uninformative equilibria do not exist or result in lower payoffs. In contrast,most players are consistently truthful in the absence of peer prediction,suggesting that these mechanisms may be harmful when truthful reporting has similar cost to strategic behavior. Finally,I design and experimentally evaluate an adaptive polling method for aggregating small pieces of imprecise information together to produce an accurate estimate of a latent ground truth. In designing this method,we make two main contributions: 1) Our method aggregates the participants noisy information by using a theoretical model to account for the noise in the participants contributed information. 2) Our method uses an active learning inspired approach to adaptively choose the query for each participant. We apply this method to the problem of ranking a set of alternatives,each of which is characterized by a latent strength parameter. At each step,adaptive polling collects the result of a pairwise comparison,estimates the strength parameters from the pairwise comparison data,and adaptively chooses the next pairwise comparison question to maximize expected information gain. Our MTurk experiment shows that our adaptive polling method can effectively incorporate noisy information and improve the estimate accuracy over time. Compared to a baseline method,which chooses a random pairwise comparison question at each step,our adaptive method can generate more accurate estimates with less cost. Abstract shortened by UMI.).