Predicting is hard. Predicting is hard because you must understand complex relationships with multiple unknowns involved. Simply imagine trying to predict your company’s next quarterly profits. Where do you start? Profits are the sum of numerous posts on the balance sheet all potentially affected by changes in internal and external factors. It’s difficult.
To make prediction problems more tractable we apply models. When we make judgments about the future, we make the problem simpler by evaluating a limited set of factors and using simplifying assumptions about how these factors interact. The very purpose of models is to simplify, which also means they make mistakes . Even the models of experts have their shortcomings and are prone to their biases, oversimplifications, and cognitive blindsided.
To make prediction problems more tractable we apply models…[but] even the models of experts have their shortcomings and are prone to their biases, oversimplification, and cognitive blindsided.
A way to bypass the cognitive limitations is to join the mental models of many people. Pooling brainpower results in richer and more accurate models (Page, 2019). This is what crowd predictions do! Let us walk through the logic and benefits of mindpooling.
The easiest way to show the logic is by the way of Page’s prediction diversity theorem. The theorem reads as follows :
Crowd error = Average error - Diversity
The equation states that the accuracy of the crowd’s prediction depends on two components.
First, it depends on the average prediction accuracy of its members. Relevant expertise on the prediction task normally results in more accurate predictions; Financial experts are normally well-suited to predict loan volume; Similarly, doctors and health specialists are better at diagnosing the diffusion of pandemics. Expertise matters.
The real value-added of crowd predictions, however, is contained in the second component. This states that the crowd’s accuracy also depends on the diversity of its members’ predictions. That’s right - diversity. A greater dispersion in predictions leads to more accurate crowd prediction.
To understand this result, which at first sight may seem illogical, it is important to recognize that prediction errors usually are distributed around the true value, not only on one side or the other. This means that when predictions are averaged the errors tend to cancel each other. An example can clarify.
Imagine two colleagues, Dora and Steve, who are both sales representatives. They’ve handed the task to forecast the number of new sales contracts that the company will successfully obtain during the next quarter. Dora predicts 20 new contracts, while Steve, being a tad less optimistic, predicts 14. The actual number of new contracts turns out to be 18.
Dora’s prediction is off by 2 contracts, while Steve’s is off by 4 contracts. The crowd prediction, which is the average of Dora’s and Steve’s predictions, is 17 and off by 1 contract. How can the crowd prediction outperform both of their individual predictions? The reason is that their predictions are situated on either side of the true value – the two predictions bracket the true value. Consequently, the crowd prediction will lie in between these two values where also the true value is located. This is called the* bracketing effect*.
The bracketing effect requires that people make diverse predictions. To obtain diverse predictions individuals must sit on different information or interpret the same information with different “glasses”. In other words, they are cognitively diverse. Another way to think of this is that they employ different predictive models. Because what extra is there to gain by two people who bring the same predictive model to the table, no matter how sophisticated it is?
However, not all types of diversity matter. For example, the cognitive diversity a random citizen brings to the table is most often irrelevant for predicting company profits. This is because expertise is equally as important as diversity.
Employees in different parts of the organization, on the other hand, do have relevant information to share. They will typically have diverse explicit and tacit knowledge about company performance that can inform the prediction task. This is where Mindpool enters the equation.
Mindpool not only helps to connect – to pool *- the organizational brainpower but to flat out engage it. Mindpool does so by enabling a seamless prediction environment that inspires and includes employees and decision-makers in challenging themselves and each other in the best of ways, giving organizations access to reap the full *diversity bonus of its vast pool of in-house knowledge .
To help tailor appropriate crowds for different prediction tasks, Mindpool assists by using data on employees' diversity and prediction accuracy. This data offers learning and improvement of predictions over time. New technologies and data analysis allow the Mindpool platform to continuously adapt to a dynamic and fast environment.
By tapping the tacit knowledge of your organization – the collective intelligence – Mindpool enables accurate predictions and dynamic forecasts. In this way, crowd predictions help to build the organizational muscle that enables more proactive decisions.
Read much more on Solutions - Crowd Predictions.
. 'All models are wrong, but some are useful' is a well-known quote usually attributed to statistician George Box.
. For those interested to explore the math behind the theorem, it can be formulated more precisely as c-θ2=1Ni=1Nsi-θ2-1Ni=1Nsi-c2 where c is the crowd prediction, si is the crowd member it's individual prediction and θ is the actual (true) value. The equation is a mathematical identity so it must be true.
. The term “diversity bonus” stems from Scott E Page’s book of the same name of which this article is heavily indebted. For anyone who wishes to dig deeper into the science of collective intelligence, his book is a perfect place to start.