The rise of prediction markets offers statisticians and social scientists the kind of help that astronomers get from a new space telescope or particle physicists from a bigger supercollider. We finally get to test theories and resolve questions that people, held back by poor data, have been wrangling over for decades.
Most importantly: Are prediction markets superior to experts and market instruments in forecasting future macroeconomic events? And can the prices on platforms including Polymarket and Kalshi Inc. guide important individual and social policy decisions?
Earlier venues that allowed people to wager on the outcomes of economically relevant events were basically laboratory studies with narrow participation, infrequent trading and low stakes. Gambling markets avoided these problems but seldom considered questions that generated data comparable to implied financial market prices or expert judgments.
With prediction markets now teeming with customers and contracts on everything from whether the US economy will be hit by stagflation to the chance of a US strike on Iran in 2026, we can finally start to understand their value — as two recent papers attempt to do by zeroing in on monetary policy forecasting.
The first, “Testing Polymarket’s ‘Most Accurate’ Claim,” considers the actions of central banks in the US, UK, European Union and Japan over the last two years and seems to point to a negative evaluation of prediction markets, though a deeper analysis shows that to be not entirely true.
It leverages the work of Wharton School professor Philip Tetlock, who gained fame 20 years ago with his book Expert Political Judgment. In it, he compiled exhaustive evidence that prominent experts were terrible at prediction, easily beaten by monkeys throwing darts or simple statistical models. The more prominent the expert, the worse the performance. The two main problems were psychological biases and poor incentives.
Tetlock exploited his knowledge of psychology and economics to offset biases and provide proper incentives, turning ordinary folk without specialized expertise or education into what he dubbed “superforecasters.” When tested using questions submitted by policymakers, Tetlock’s superforecasters blew away not only other academic teams but also the professional forecasts made by highly trained specialists with access to classified intelligence.
But how do superforecasters fare against prediction markets? In “Testing Polymarket’s ‘Most Accurate’ Claim,” the authors used an academic criterion known as “Brier score” to judge accuracy and found superforecaster teams had a better (lower) score than Polymarket prices.
Most Bloomberg readers will probably find a different metric more meaningful. If you bought every Polymarket contract for which the superforecasters thought the probability was greater than the contract price and sold every contract where the superforecaster probability was lower than the price, you’d have made $477 over the 6,393 one-dollar bets, a 7.5% edge.
Before deciding that superforecasters know everything Polymarket traders know and more, I dug a little deeper and calculated the actual frequency of events versus probability predictions. The blue line is perfect prediction — events predicted to have 50% probability happen half the time and so on.
Both superforecasters and Polymarket are bad in the tails and better — though far from perfect — around the center. Events predicted to have under 20% probability almost never happen, while those with over 80% probability almost always happen. Polymarket is best with prediction probabilities between 35% and 65%; superforecasters are better between 20% and 35% and 65% to 80%.
What if we consider the optimal combination of the two forecasts? My calculations put 60% of the weight on the superforecasters and 40% on Polymarket. Superforecasters beat Polymarket, but the combination beats both individual forecasts. Polymarket’s prices contain significant information that can be used to improve superforecaster estimates.
In “Kalshi and the Rise of Macro Markets,” published by the Federal Reserve, the authors tackled a different question, one of utility. Unlike traders who want to make money betting on macro trades or decision makers making plans that depend on macro factors, the Fed wants to manage macroeconomic expectations.
Their main finding is Kalshi provides much richer information about the distribution of expectations, not just what the average or median market participant expects. After all, the economy is driven by billions of decisions by hundreds of millions of individuals, not the average or median person and not just a few highly paid Wall Streeters.
The researchers also found that Kalshi prices have comparable accuracy to the New York Fed Survey of Market Expectations as well as Bloomberg’s consensus forecasts; it even beat the Bloomberg consensus for the consumer price index. Kalshi is particularly good — or experts and financial markets are particularly bad — at quantifying tail risks. These are more important to the Fed than the everyday ups and downs of markets.
Even if the Kalshi predictions were not this accurate, they would be useful in helping the Fed understand what people think. Moreover, since Kalshi prices are continuous and intraday, they allow the Fed to see the immediate effect of single events — an announcement, a press conference, a speech. The paper confirms that these rapid responses are meaningful.
While the paper is limited to the perspective of central bankers, we could expect similar results for all decision makers concerned with public opinion. Some obvious fields are election analytics, real estate development and investment, fashion and entertainment, and social and news media.
In one sense, both papers are perishable. Prediction markets are evolving rapidly, and artificial intelligence is coming up in the rear review mirror. With more trading volume, contracts, liquidity and users, prediction markets should up their game. But AI is nipping at their heels already. While AI prediction algorithms have not yet matched superforecasters, they’re moving up the leaderboard rapidly. In fact, an elite team of human forecasters at online prediction platform Metaculus puts a 95% probability on AI beating them — and all other humans — in forecasting by 2030.
A message from Advisor Perspectives and VettaFi: Discover something new! Click here to register for our upcoming webcasts.
Bloomberg News provided this article. For more articles like this please visit
bloomberg.com.
Read more articles by Aaron Brown
