by Dan White
Of the many cognitive biases and heuristics that affect decision making, none may be more common or powerful than confirmation bias. Confirmation bias is the tendency to seek out evidence that supports a point of view and discount or outright discard evidence that does not support that point of view. This can present itself in many different situations. From reviewing data on the performance of a recent promotion, reviewing guest comment cards or market research to assessing the findings from a feasibility study or a proposal from a vendor, confirmation bias can play a role in how we assess information and make decisions.
Considering it can impact decisions and considering many of those decisions may represent significant investments of capital and resources, our ability to diminish the impact of confirmation bias can help avoid serious errors in decision making and operational performance.
In 1960, psychologist Peter Cathcart Wason designed an experiment to demonstrate cognitive errors in human reasoning. The experiment, called the 2-4-6 problem, featured subjects who were given a series of three numbers and told there was a specific rule the three numbers followed. Subjects were instructed to try to discover the rule by offering their own series of numbers. They received immediate feedback as to whether the number set they offered conformed to the rule or not. But many could not successfully discover the rule. In about 80 percent of the cases, subjects offered numbers that sought to confirm their theory and very few attempted to disconfirm, or falsify, their theory.
The rule, in this case, was any three ascending numbers. Any combination they offered of three numbers in ascending order conformed to the rule but most subjects could not determine this rule. In many cases, the subjects grew increasingly convinced their theory was correct, despite the fact that it was not. This is a manifestation of confirmation bias.
Wason, who would later coin the term “confirmation bias,” was on to something; we tend to seek evidence that supports our beliefs and reject evidence that contradicts those beliefs. And it is, in many ways, hard-wired into the human mind; we do it without thinking about it.
Take, for example, a feasibility study to determine the potential demand for an extensive casino expansion. Such a project is likely to create a lot of excitement and optimism. But what if the feasibility findings are inconclusive or mixed? If you have already formed an opinion about the project, it is more likely you will find evidence in the study that supports your position. Perhaps you point to the estimated additional revenue and overlook the fact it will take more than five years to see a return on the investment. It is entirely possible that two people could review the same study and walk away with different opinions of the project. In some cases, those differences are driven by confirmation bias.
It is crucial to understand the impact this can have on an organization, particularly one that is risking millions of dollars on the outcome of a project. So how can the influence of confirmation bias be mitigated and minimized?
One concept that may help is known as falsification. Championed by 20th century philosopher, Karl Popper, falsification is at the heart of the scientific process. It is, in essence, the opposite of confirmation bias. It is approaching our knowledge with skepticism and attempting to disconfirm our theories and points of view.
Where confirmation tends to build knowledge on a series of observations that, over time, lead to hypotheses and theories (a form of inductive reasoning), falsification begins with a theory and then sets out to falsify, or disconfirm, the theory (a form of deductive reasoning). The problem with inductive processes is they may never yield an accurate theory no matter how many observations, or confirmations, are made. One of Popper’s most famous examples of this was the black swan problem. Up until the time black swans were discovered in Australia, the belief was all swans were white. It took likely thousands, if not millions, of observations to arrive at this theory but only one observation of a black swan to completely discredit it. This demonstrates the power of falsification. There are some practical and tangible ways to cultivate a falsification mindset. In fact, many of the steps you can take rely on simply aligning existing processes and a small adjustment to the thought process.
Start With a Theory
A theory is the primary ingredient in falsification. Without it, there is nothing to falsify. In the case of casino marketing, think of this as your pro forma. A pro forma establishes the expected performance or outcome of a project, program or promotion. This is what you will attempt to falsify as you measure the outcome of the promotion. For example, perhaps you create a promotion designed to increase revenue on Wednesdays by 12 percent. Additionally, you believe the promotion will increase total rated players by 15 percent and minutes played by an average of 18 minutes. Be clear and specific where you can. This is the theory you will now measure against.
Establish Clear Key Performance Indicators
Similar to above, you should identify the key metrics you will use to measure the results of the program or promotion. Casinos, and casino marketers in particular, are buried in data. It is entirely possible to tell multiple stories about the success (or failure) of a program with all the data that is available to marketers. In this case, we might determine that the key performance indicators (KPIs) for the Wednesday promotion mentioned above will be gross revenue, rated play and minutes played.
Avoid Post Hoc Fallacies
One of the reasons an organization needs a clear theory and well-defined KPIs is to avoid post hoc fallacies from forming. A post hoc fallacy is when stories are created around results after the fact. In many cases, this can lead to finding correlations that don’t exist or, perhaps worse, mistaking correlation for causation. Without KPIs and clear theories, we are left with all that data and can build a number of narratives through these observations that ultimately do not get us closer to the truth.
Introduce Rigor to the Process
Once the program or project is underway, it is time to introduce rigor into the process. In this case, rigor represents a diligence to objectively measure the results. Since a clear theory and clear objectives have already been established, a well-defined path can exist. Did the project or promotion produce the results the theory suggested it would? If so, this provides some initial support for the theory, but is not conclusive. Additional testing should continue. If it did not meet the predictions, a new theory is in order and the process will repeat itself.
Be Curious
When evaluating any information, maintaining a sense of curiosity can be crucial. Ask questions. What is the size of the sample population (sampling errors). Is this information complete or is it partial (cherry picking)? What information is not included that may otherwise be of value (survivorship bias)? These types of questions can help you avoid the traps of confirmation bias and help you adopt a more objective and skeptical mindset.
While it is unlikely to eliminate confirmation bias completely, it is possible to contain it. This is particularly key when making important decisions. Introducing falsification and maintaining some measure of skepticism in our knowledge can help protect us from the risks of confirmation bias.
Dan White is Director of Marketing for Muckleshoot Casino in Auburn, WA. He can be reached by calling (360) 890-1433 or email [email protected].