Malcolm Gladwell took the stage to present next. Malcolm is the author of some great books (such as Blink, reviewed here, and Outliers). Gladwell began by noting that Vegas seems an odd place to hold a conference on analytics!
His first story was about art – a dealer brings a new statue to the Getty, a Kouros, and they were excited but wanted to check its authenticity. The lawyers go first and find no problem with the paperwork. Next they check the stone to see if it comes from the right quarries and see if it has been out of the ground long enough. So far so good. They decide to buy it, pay a huge amount of money ($10M) for it and invite an expert to see it. She takes one look and says its a fake but can’t say why. Another expert, same immediate response but not able to say why. They go to Greece and unveil it to a huge audience of experts and have the same response. When they get back, the lawyers call to say there is now in fact a problem with the paperwork and the geologists call to say there is a problem with the age test. In the end, then, the data shows it is a fake just like the experts thought it was.
So, in a business, how do you create conditions so your decision makers can act based on their expertise, their judgment when it matters?
In the statue example the experts were doing a kind of complex pattern matching – taking a pattern they had in their head about real Kouros statues and matching it to the actual example in front of them. This pattern recognition – rapidly seeing the pattern in the real world and knowing what to do. In military circles they talk about coup d’oeil – at a glance – the ability to see immediately what was needed. Building this requires massive amounts of experience with seeing the pattern. And this is backed by research that says you need 10,000 hours of experience – say 4 hours a day for 10 years. It is very hard to find people who are top of their profession without this 10 years of experience. So what is going on to let these folks become experts in this 10 year, 10,000 hours? Lots of feedback is what happens – try, fail, get feedback, learn, repeat.
Lesson #1, then, is that tools that help improve the quality of feedback we can increase the value of experience in building judgment.
Judgment, though, has some drawbacks. In particular it is mysterious. When Gladwell spoke to the various experts who said the statue was fake they had no idea why they thought it was fake. They did not make their judgments lightly – this was a big deal, reputations were riding on their statements. Even so, they still did not know why they made this critical judgment. This is typical of judgmental decisions – lots of use of our unconscious. This reliance on the unconscious makes it hard to explain why they were made. And this is true of everyone from a tennis coach who could spot double faults before they happen every time to tennis players who cannot explain how they hit topspin forehands. Their knowledge has been transferred to their unconscious so they can’t really explain, only give a plausible explanation (that may well be wrong).
Lesson #2 is that decision making tools can help by exposing and making transparent decisions so that the knowledge can be taught and shared.
What about the Getty? Why did they buy the statue. They had staff members who were experts in statues of this kind, including one of the world’s best. So why didn’t they call the fake what it was? Part of the problem is that the Getty can only buy art from before 1900. This, of course, is hard so there is a desire to find art to buy and in this case there was an emotional desire to believe the statue was real. This kind of emotional investment can seriously undermine judgment. On a similar vein he looked at a group of appointed CEOs. Overwhelmingly these CEOs, who didn’t found the companies, were tall, white men. And really tall, much taller than average. Why are boards picking these tall men as CEOs well because tall used to be an advantage for savanna hunters and this desire for tall leaders lurks in our subconscious and distorts judgment. In legal issues, black criminals are punished more severely than white criminals who commit the same crime. Bias exists and distorts judgment in many areas. Judgment is very fragile.
Lesson #3 is that judgment is weak and needs tools that protect judgment from corruption and bias.
Last section. Think about the Getty who spent 11 months to gather tremendous amounts of data but the experts just show up and overturn this, get a better result. Unaided human judgment tends to work best where there is just a little bit of data – a glance at the statue, for instance. Getty knew too much and it swamped their ability to apply their judgment. For instance, doctor’s do a poor job of diagnosing heart attacks in patients coming to the Emergency Room. Analysis of results shows that insisting on doctors asking just a few, pertinent questions (4 in fact) then their decision making improves dramatically. The analytics have clarified the field of decision making. All to often the problem with decision making failures, in military intelligence for example, can be traced to too much information. For instance, before Pearl Harbor or 9/11 there was so much information available to analysts that they could not see the pattern.
Lesson #4 is that systems must clarify and focus on the information that will matter so that decision makers can act on a reasonable number of data elements.
He finished up with a story of a female trombonist who was mistakenly invited to audition (they thought she was a man) and who then auditioned behind a screen. The conductor is sure she is the best and is shocked when he sees that she is a woman. Like most conductors at the time he was convinced that men made better players than women and he had a 100% male orchestra. He had been picking less good players because of this bias. And as screens go up for other auditions, in the early 90s, more and more women are taken into orchestras changing the percentage from 2% to 50%. Until screens went up these people had been unable to make a good decision because of their bias. With a screen, all you get as input is their playing. Without a screen the visual data, that does not matter in selecting a musician, overwhelms the process.
To get the best decisions from your decision makers you must build tools to nurture and protect those decision makers.
Gladwell spoke without notes, without slides and was very engaging.