Some people have a gift for predicting the future. Not a vague, ambiguous prediction, but reasoned, cautious and thoughtful foresight. These people can see the likelihood of a companies’ commercial success or the outcome of elections better than anyone else. They are called “super-forecasters”, and what they can teach us about how to make smarter judgements could save companies billions, or even prevent countries from going to war.
In 2007, Steve Ballmer, Microsoft’s then chief executive, told USA Today that “there’s no chance that the iPhone is going to get any significant market share”. He later went on to predict Apple might take 2-3% of the growing mobile market. Ultimately, Apple’s global market share peaked at about 23% in early 2012. Ballmer made bad predictions. (This article is adapted from an episode of CrowdScience from the BBC World Service. You can listen to the whole programme by following the link – Can I predict the future?).
Ballmer did not possess the characteristics of a super-forecaster – humility, open-mindedness, inquisitiveness, among other things. What made things worse was his lack of willingness to amend his forecast. The people who make the best predictions about the future are also happy to change their prediction when presented with new information. Ballmer did not, and Microsoft’s presence in the smartphone market suffered as a result. Some critics have described him as “the worst CEO of a large public company” in the US.
The devastating effect the earthquake and subsequent tsunami had on the Fukushima Daiichi power plant were largely unforeseen by the Japanese authorities (Credit: Getty Images)
If we know what makes someone good at predictions, can we teach people to be better forecasters? Perhaps, if it is possible, mistakes like Ballmer’s could be avoided.
Good judgement
The idea of cultivating super-forecasters has only fully been investigated in the last few years. From 2011 to 2015, the Intelligence Advanced Research Projects Activity (IARPA), which is part of the US Office of the Director of National Intelligence, ran a tournament to find teams of naturally talented forecasters. The payoff for the US intelligence community here was potentially huge: a large, organised group of forecasters who could make significantly smarter predictions than their own intelligence officers.
In total, 25,000 forecasters took part, making predictions on the future of the Eurozone to the likelihood of Vladimir Putin losing power in Russia. Forecasters who worked in teams outperformed even well-trained individuals – the theory being that each member balances out any biases in the other members. The winning team, the Good Judgment Project, subsequently became a forecasting business. (Read about the simple rule that can help you predict the far future.)
Leicester City started the 2015/16 Premier League season as 5,000/1 outsiders to win the title – few would have predicted their remarkable success (Credit: Getty Images)
“If you look at some of the media or political experts at the moment – watch the news – everyone is saying ‘This is unpredictable, I don’t know what will happen’,” says Kathy Peach, who leads Nesta’s Centre for Collective Intelligence Design. “If you open a newspaper you’ll see those incredibly complicated flow charts of all the different Brexit outcomes. So, if the experts are saying they can’t predict what will happen, if the statistical methods of prediction don’t work because there is no historical precedent for this, then how are you meant to make decisions either as an individual or as an organisation?”
The Centre for Collective Intelligence Design is part of Nesta, the UK innovation foundation, who are collaborating with BBC Future to find individuals who can forecast global events and to investigate their predictions. (Read more about the challenge has gone so far here).
A light aircraft flies close to Eyjafjallajökull during the 2010 eruption which caused the highest level of air travel disruption since World War Two (Credit: Getty Images)
“In a time of increased complexity and uncertainty it’s actually increasingly less likely that any one single individual will have access to all the information about what is happening,” says Peach. “But if you take a collective view by bringing together and combining the predictions of lots of different people you get to a more accurate result because they all hold different pieces of information that help to build a more complete picture overall. By combining those individual forecasts you’re also cancelling out, perhaps, some of the biases or inaccuracies that might exist in one individual forecast alone.”
Pattern recognition
The Good Judgment Project’s team of super-forecasters repeatedly made excellent predictions – so what set the individuals in the team apart from the rest? As an original member of the team, Michael Story is one of the world’s best super-forecasters. Now, he is managing director of Good Judgment Inc, the commercial spin-off of the project. “When we test people to see whether they are likely to be a good forecaster, the number one predictor of forecasting isn’t subject knowledge or anything like that, it is pattern recognition from pictures,” says Story.
By asking people to spot patterns in a series of photographs you can make an accurate assessment of whether someone is a good forecaster. But to do this, they need to be able to see past their own biases.
Story describes confirmation bias – where we selectively look for evidence to support our own ideas – as being like playing cards. How many times do you or other players say “Oh I knew you had that card” after someone reveals a winning hand? You feel certain that you knew they had those cards all along, but in reality you thought through several possibilities. When one of those is revealed you convince yourself that that was the possibility you felt most strongly about.
“I always thought I was quite good at predicting things and so on, but it is also extremely easy to convince yourself of that fact,” says Story.
Philip Tetlock, one of the founders of the Good Judgment Project describes super-forecasters as being “somewhat distinctive psychologically”. They possess a unique combination of characteristics that enable them to overlook their own prejudices.
“I think that if I had to identify one particular thing, it is that whereas most people think of their beliefs as something very precious and self-defining, even sacred sometimes, super-forecasters tend to see their beliefs as testable hypotheses that should be revised in response to evidence,” Tetlock says. “That means they tend to be better belief-updaters... as news comes in and requires either moving a probability up or down.”
Few would have predicted Olivia Colman would beat the frontrunner, Glenn Close, to the best actress Oscar in 2019 (Credit: Getty Images)
Super-forecasters have also been observed to make smaller changes to their predictions than other people, when given the opportunity to revise them. It might be the case that their original estimate was already pretty accurate. But researchers argue that super-forecasters can think of many more possible outcomes than an average forecaster. When they revise their answer they move to a slightly different response rather than jumping wildly to a completely different solution.
Research suggests that groups of women are more collectively intelligent than groups of men
Peach says she is also keen to explore whether women make better group predictions than men. Similar research suggests that groups of women are more collectively intelligent than groups of men but further work is needed to see whether collective intelligence equates to super-forecasting. The more intelligent you are, for example, the more likely you are to be guilty of confirmation bias, as you are better at finding reasons to support your own argument.
Should you wish to improve your forecasting skills you will be pleased to hear that super-forecasters are partly discovered and partly created, according to Good Judgment Project co-founder Barbara Mellers. The more predictions you make, the better you get.
BBC
More about:
