The following is an excerpt from Moving at the Speed of Business, a new report that explores a range of methods that market researchers can use to quickly and simply quantify information to build confidence and incite action across the company.
Research shows that, despite rigorous testing, the majority of product launches fail and that consumers are notoriously unreliable in accurately reporting on their future purchase behavior. The value of asking standard “intent to purchase” questions is, well, questionable.
In contrast, when you ask a diverse group of people not, “What are you going to do?” but rather, “What is going to happen?” the results are much more accurate. This is the fundamental approach that underlies prediction markets, which incorporates “gamification” principles into the concept development process. Prediction markets work similarly to the stock market, where people — or players — are allotted points to invest in product ideas they believe will be winners in real life. They anticipate outcomes: the likely success or failure of an idea, concept, or product, for example. Each participant in the market is given play money or points to invest in their answers. They answer only the questions about which they have a strong opinion; can invest in the likely failure of an idea as well as in the likely success of one; and invest as few or as many points as they want based on the strength of their confidence in their own predictions. Furthermore, when they invest, they provide an explanation of why they’re doing it. When the market closes, each possible outcome ends up with a numeric value representing the probability that it will come to pass, as well as with a “strength meter” score reflecting the passion of people’s convictions based on the number of unique investors, the total points invested, and the average investment. As illustrated in the example below, the result is a far more nuanced read on what’s potentially polarizing, and on where investors’ passions lie.
Research, and C Space’s own experience running these markets, has shown this approach outperforms asking consumers to self-report on their own future behavior. Additionally, these markets can be used not just with consumers, but also with employees, with equal success.
For example, we ran a prediction market for one of our clients (a media and gifts company) to test five products they had launched during the holiday shopping season. All five products had performed well in traditional quantitative studies; the question we posed in our markets was whether or not “investors” thought the new versions of these products would outperform those from the previous year. This exercise allowed us to test how well the prediction market could replicate what happened in the real world. It also helped us understand if employee-based markets would fare as well as consumer-based ones.
The results? Our client compared the prediction market output to actual sales data, and found the consumer market accurately predicted outcomes for three of the five products, and the employee market actually achieved 100% accuracy (getting it right in all five cases).
When we allow people to put their money where their mouths are, we get a more accurate picture of what is likely to succeed or have value in the marketplace. Prediction markets generate sound and useful results when run with small sample sizes in online communities.