Eric Siegel and I had a great discussion about doing Machine Learning BACKWARDS recently – you can watch the recording below or on our YouTube Channel. Eric, if you don’t know, is the founder of Predictive Analytics World, a leading consultant, and author of “Predictive Analytics“. You can also check out Eric’s new Coursera class.
This discussion was prompted by Eric and I talking about the rate of failure in Machine Learning projects. For instance, one survey said that 85% or more of machine learning projects fail to add business value and that number has gone up, not down, in recent years. Our premise is that the best way to avoid these failures is to do machine learning backwards – to begin with the outcome you want, an improved decision, and work back to the models you need and the data that will let you build them.
At the end we took some questions and one of the questions we got was:
How do you recommend getting senior executives engaged?
First, we said, you need to focus the discussion on the value from deploying a solution not the core technology. This means you might want to avoid using the words “model” or “predictive model” or “machine learning model”. Instead, focus on is exactly which decisions within which large scale operations are going to be improved and to what degree they could potentially be improved. Then you can start to talk about probabilities such as that these people are much more likely to cancel and how these probabilities are going to help make decisions more profitably. After all, you can place customers into at least two very different groups based on those probabilities and treat them accordingly generating differentiation.
I discussed one useful exercise we have done with executives. We start by asking them how they are measured – how they measure their own personal success, which metrics they care about because those metrics drive their bonus. Then we’ll ask them to identify the decisions that get made in the organization that have an impact on those metrics. The first few are always big strategic decisions that the executive team make.
If you keep pushing on it, though, gradually they realize that there are decisions made by all sorts of people in the organization and indeed by bits of software infrastructure that matter to the metric also. And while they trust their own judgment – they don’t need analytics – they are much less sure about the judgment further down the organization or in the IT department. Once they realize that machine learning is not about improving their personal decisions but about improving the quality of decision making at the operational frontline they get much more excited.
Machine learning teams often feel pressure to make a strategic difference to the company. They mistakenly assume that the way to do this is to have machine learning influence the company’s executives and executive-level decisions. This is a mistake. Better, instead, to work with executives to find the high volume, repeatable decisions that make a difference and use machine learning to improve them. Because these decisions are made so often, even small improvements multiply to give you a strategic impact.
Lots more good tips in the video. If you are interested in how we approach this why not read our white paper on Framing Analytic Requirements.
