Cassie Kozyrkov, the Chief Decision Intelligence Engineer at Google wrote an article recently titled Is your AI project a nonstarter in which she identified 22 check list items for a candidate AI project. It’s a great article and you should definitely read it. In particular you should note the quote at the top:
Don’t waste your time on AI for AI’s sake. Be motivated by what it will do for you, not by how sci-fi it sounds.
And what it will do for you is often help your organization make better decisions.
We always begin customer projects by building a decision model. Working directly with the business owners, we elicit a model of how they want to decide and represent it using a Decision Model and Notation (DMN) standard decision requirements model. This shows the decision(s) they want to make and the requirements of those decisions – the sub-decisions (and sub-sub-decisions), the input data and the knowledge sources (policies, regulations, best practices and analytic insights) that describe their preferred approach.
These models address several of Cassie’s early points (1. Correct delegation and 2.Output-focused ideation) by focusing on the business and on business decision-making. We also link this decision model to the business metrics that are influenced by how those decisions are made, which addresses couple of her key points on metrics (18. Metric creation and 19. Metric review).
This decision model is often a source of analytic inspiration, as business owners say “if only…”- “if only we knew which emails were complaints”,” if only we knew who had an undisclosed medical condition”, “if only we knew if this text document described the condition being claimed for”…. These are the analytic and AI opportunities in this decision. Like Cassie, we often find that existing data mining and description analytics projects can be used to see how a decision could be improved with AI/ML (3.Source of inspiration).
Now the decision model has sub-decisions in it that are either going to be made by a person or by an AI algorithm. Because you know what a better decision looks like (thanks to the link to business metrics), you can make sure an AI algorithm is likely to help (20. Metric-loss comparison) and you can consider if the specific decision you identified is a good target for AI (4. Appropriate task for ML/AI). Critically we find that often the whole decision is not suitable (there are too many regulations or constraints) but critical sub-decisions ARE suitable.
When it comes to putting the resulting AI algorithm or ML model into production, the decision model makes it clear how it will be plugged in and how it will be used in the context of the business decision (5. UX perspective and to some extent 8. Possible in production). Keeping the end – the decision – in mind in this way means that project teams are must more focused on how they will operationalize the result of the algorithm than they would be otherwise.
If you automate the decision model, as we do, using a BRMS then you will also be able to simulate the decision against historical data (17. Simulation). The decision model means you can simulate the decision with and without your AI/ML components to prove the ROI too.
Finally, this focus on decision-making means you know when the AI/ML model will be used (other sub-decisions are likely to address eligibility and validity of the transaction, for instance, narrowing the circumstances in which the AI must work) and you can see what accuracy is required. This is often much lower than AI/ML teams think because the decision model provides such a strong frame for the algorithm. (21. Population and 22. Minimum performance).
Decision models are a really powerful way to begin, scope, frame and manage AI and ML projects. Of course, they don’t address all Cassie’s 22 points and the others (6. Ethical development, 7.Reasonable expectations 9. Data to learn from, 10. Enough examples, 11. Computers, 12 Team, 13 Ground truth, 14 Logging sanity, 15 Logging quality, 16 Indifference curves) will need to be considered decision model or not. But using a decision model will help you frame analytic requirements and succeed with AI.
