Study: Experts aren’t necessarily better forecasters than ordinary Joes

There’s a geopolitical conflict, or a labor dispute. You’d think that long-time experts in the field — who’ve seen this kind of thing time and time again — would be the best at predicting the outcome of those disputes. But they aren’t.

“The short answer is that they [expert predictions] are of little value in terms of accuracy. In addition, they lead people into false confidence,” says Kesten Green of Monash University in Australia.

The forecasts of experts who use their unaided judgment are little better than those of novices, according to a new study in a publication of the Institute for Operations Research and the Management Sciences (INFORMS).

When presented with actual crises, such as a disguised version of a 1970s border dispute between Iraq and Syria and an unfolding dispute between football players and management, experts were able to forecast the decisions the parties made in only 32% of the cases, little better than the 29% scored by undergraduate students. Chance guesses at the outcomes would be right 28% of the time.

Moreover, the study’s authors say that relying on “expert predictions” discourages decision-makers from investigating alternative approaches.

However, remember that the results above were based on experts’ “unaided judgment” (translation: off-the-top-of-the-head opinions). The researchers say that experts can improve their forecasts by using “reliable decision-support tools,” such as:

  • simulated interaction, a type of role-playing for forecasting behavior in conflicts, which reduced experts’ forecast errors by 47%
  • structured analogies, which reduced experts’ forecast errors by 39%.

More information on this topic is available at ConflictForecasting.com.

———-
Source: “The Ombudsman: Value of Expertise for Forecasting Decisions in Conflicts,” by Kesten C. Green of Monash University in Australia, and J. Scott Armstrong of the Wharton School at the University of Pennsylvania. It appears in the INFORMS journal Interfaces, Vol. 37, No. 3.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s