Knowledge Stacking: a crowd in search of the truth

September 2017

The wisdom of the crowd lies at the heart of democracy and is a paradox which has mystified us for as long as it has intrigued us. 21st century access to crowd-sourcing techniques means that, for the first time, the sum total of human knowledge is within reach; but how has knowledge ever helped us to find the truth? Exploration for natural resources is a commercial endeavour which is ground-truthed in reality: the business takes a chance on a concept, which is based on how an opportunity is capable of making money, then we test that concept by drilling. There are not many scientific or artistic endeavours, where we get an absolute measure of the force of our ideas. We tell our investors that the more we know, the greater the likelihood that our selection of the truth matches reality. Somehow, it never quite works like that…

Somewhere between concept and drilling lies a vast array of potential forecasting outcomes (scenarios), which we weigh against our own cognitive biases; our versions of the truth. Investors abhor scenarios. How often have you heard a manager say “yes but what do you (as the technical expert) actually believe?” and how often have you obliged them with a single answer to prevent yourself from appearing indecisive? Investors make a decision to invest on a singularity because it is much easier for them to bet on a single outcome than on multiple scenarios. A singularity because everyone knows that there is only one truth and this is it, basing their choice on what is a belief supported by your technical testament. A belief that your selection of the most likely outcome matches reality (no pressure).

The problem is one of perspective – our anterior versus posterior view. Anterior forecasting generates multiple versions of potential outcomes and many of them are equally as likely. The business runs into difficulty when our speculative belief in the singularity (aided by our cognitive bias) anchors us and renders us perceptually blind. In other words, we either choose the wrong potential outcome or worse: we overlook the most likely outcome, which is staring us in the face. 

THE SIMULATION

K2V Ltd has devised a simulation to test whether “stacking*” knowledge, which is commonly informed but diversely perceived, can preserve diversity of opinion on the one hand whilst collapsing cognitive bias on the other, leading to a more realistic set of outcomes. The simulation invites students to play the role of an exploration manager in evaluating a new ventures opportunity using a predefined set of metrics. The students, who may not have a geoscience background, will be given a brief introduction into what drives exploration growth. The introduction provides them with a reference frame, defining petroleum system robustness and the business context for success, effectively anchoring their perceptions in the same way, with the potential for “GroupThinking”. Each student will then be invited to assess a real opportunity for an investment decision. The opportunity will be presented and followed by a live questionnaire, which crowd-sources their independent opinions on 15 compressed geo-technical and geo-commercial (scorable) metrics. In the process, each student will be asked whether they would invest in the opportunity and what sort of mood they were in when they responded to the questionnaire. The results will be compiled as an abstraction of their estimation of the value, graphically presented on a similar cross-plot to the one below. The scale has already been calibrated for anterior and posterior views by four subject matter experts who have assessed the opportunity using the same independent simulation and by the fact that the opportunity in the real world has now been drilled.

The crowd-sourced cross-plot will be shared with the students at the event so that they can examine their own collective opinion, making them self-aware in a way that they weren’t when they initially responded to the questionnaire. If there is time, the survey will be rapidly repeated to see if that self-awareness as a group influences their choices as independent individuals, potentially altering the range and the bias.

The simulation is an experiment to establish whether a sample of individuals supplied with the same information can independently render a credible estimate of value irrespective of their experience or knowledge. The results of the experiment are as much a test of the granularity of metrics used as they are in collapsing cognitive bias from an unfiltered pool. At the very least, the simulation will capture the full range of perceptions; at best, it will converge on a common understanding of reality. At this stage, however, it is not known whether informed opinion can be abstracted to identify the most likely outcome or whether the stacked view will just depict random opinions that only reveal biases.

The pilot simulation will take place this autumn at the first 4 of 12 university geology faculties around the UK who have expressed an interest in participating. Successive simulations will be adjusted for pitch (change the spin) and assessed for stability. The resulting metadata will be compiled, examined for content, filtered by levels of knowledge and processed to pull out competing hypotheses. Whether the metadata are processed arithmetically or by applying Bayesian approaches (GoExplore) has yet to be established. Point estimation of a discrete continuum is demonstrably resolvable but overlaying individual cognition with overlapping and typically competing hypotheses to quantify unique systems is practicably unachievable. It remains to be seen if the metadata from the simulation can generate an improved understanding of the effectiveness and dynamics of crowd-sourcing, and its potential to assist in decision making.

THE RESOLUTION

Of course, however democratically pleasing it may be to imagine otherwise, even though none of our anterior views are wrong, we can’t all be right. This is where crowd-sourcing becomes difficult to use because we are generally unable to weigh anterior opinion in favour of a forecasted reality. Crowd-sourcing geo-technical opinion to bid for acreage has been successfully pioneered by Draupner Energy in the North Sea and has been developed further in other endeavours like exploration for gold. The challenge is how to collapse unwanted cognitive bias or at least mitigate against its erosive effects on decision making (Rose & Associates). K2V has always recommended a collaborative approach by having discussions to form a common opinion; collapsing bias through a free and honest exchange of views. However, some organisations are rigorously siloed, sometimes tax ring-fenced to specifically exclude the cross-flow of information or even views. Under those circumstances, the only common meeting point is at the board level where investment decisions are being made but where the content knowledge is typically under-represented. The simulation polls a student population with low industry experience and similar levels of knowledge. One individual may have fifty experiences once and another, one experience fifty times but the quality of perception is how that individual maps his/her learning on to the specific objectives that the knowledge is being applied to. If stacking knowledge as a technique works, it could help to change the way investment decisions are made in the future.

Psychologists tell us that cognitive bias is an “irrational error in thinking” and should be mitigated against to achieve a rational viewpoint. K2V sees cognitive bias as the subjective but nonetheless coherent culmination of years of experience with a richness in the diversity of perceptions that can be used to explore the dimensions of truth which makes our knowledge so valuable to decision making. We should not be screening wild ideas out of our thinking; we should make them work for us without misleading us. It may not be possible to identify the unique set of circumstances that contribute to cognitive bias but if we all thought the same way e.g. GroupThink, by definition there would be no opportunity. Crowd-sourcing reminds us that there is no anterior singularity (no pre-determined truth) and keeps us honest by opening the aperture wide on possibilities. It prevents us from losing sight of the upside potential whilst we protect our business from the downside.

K2V Ltd would like to acknowledge contributions for this article made by Frank Glass, Paul Cleverley and Simon Neal, without whom all of the above would have remained just an interesting idea.

*”Stacking” is a term borrowed from geophysics, where offset viewpoints acquired at the surface are stacked to a common depth point as a single representation of the full collective range of offset views.