KDNuggets had an interesting poll this week in which readers expressed themselves as Skeptical of Machine Learning replacing Domain Expertise. This struck me not because I disagree but because I think it is in some ways the wrong question:
- Any given decision is made based on a combination of information, know-how and pre-cursor decisions.
- The know-how can be based on policy, regulation, expertise, best practices or analytic insight (such as machine learning).
- Some decisions are heavily influenced by policy and regulation (deciding if a claim is complete and valid for instance) while others are more heavily influenced by the kind of machine learning insight common in analytics (deciding if the claim is fraudulent might be largely driven by a Neural Network that determines how “normal” the claim seems to be).
- Some decisions are driven primarily by the results of pre-cursor or dependent decisions.
- All require access to some set of information.
To ask if one kind of know-how will replace another seems to me, then, to be the wrong question. Better to ask if the balance between manually documented know-how and machine learning will change and, if so, where and why? We could also ask if there are really any decisions where machine learning or analytics cannot help at all (probably but only because the decision-makers don’t have access to data that would help or because they are obliged to follow a precise set of regulations/policies). Or we could ask if there were any decisions that only required know-how that can be derived automatically using machine-learning (probably not, most business decisions involved some policy and regulations that are fixed even if we can replace experience with machine learning).
Too many analytic professionals think that only the data speaks and that business rules are, as someone once said to me, “for people too stupid to analyze their data”. Similarly too many IT professionals think that everything can be reduced to business rules or to code using explicit analysis. The reality for most decisions is somewhere in between.
Not machine learning or domain expertise but machine learning AND domain expertise. Decision Management in other words.
Comments on this entry are closed.
I disagree, but partly. A machined learned output would define an outgoing procedure. Say what if you have a stable system, yet it does not achieve the output you liked. Then you would do a process analysis, devise a means to get to the target, AS-IS to TO-BE. What you say as AND between ML & DE, that can be dealt with with exceptions to those identified process.
I am not talking a situation where something is being done from scratch but systems that are already in place, they can be process analyzed and the already present machine learned expertise shall help in detailing the required output you would have liked.
I have worked on 4 different domains and have process analyzed and moved systems in this fashion to these required output targets. I don’t see how domain expertise might play a role. I can agree to one fact though, any change in the external of a system can only be fed into the system after it is understood by a domain expert as to what the change may have been and an impact of it on present system. Hence the part disagreement. Feel free to continue this debate.
Using machine learning to improve an existing process is clearly an effective use case. However the result of the machine learning must still be fed into a decision of some kind. Some of those decisions can and should be made purely based on the machine learning output, some can be made using a mixture and sometimes the right thing to do is filter the machine learning conclusions through a human expert and then use them. Lots of combinations that make sense for different decisions but the balance is generally there somewhere.
I Agree
James, of course Machine Learning can benefit from domain expertise. However, the question I raised whether Machine Learning on Big data can exceed the human expertise, at least in the area of prediction and classification? Not today, not next year, but perhaps in 10, 20 or 100 years. We are seeing more and more examples of it today – just check any recent analytics competition, where machine learning experts increasingly beat results obtained by domain experts. For example, in http://host.kaggle.com/casestudies/allstate the winning entry was 340% more accurate than domain experts existing method for predicting car insurance claims.
Gregory
I don’t disagree that we have many decisions today made by people or based on explicit human judgment embodied in business rules or code where machine learning could improve the quality of decisions. But even where this is true there is often still a need for human judgment. Regulations and company policy can be good reasons for overriding the machine learning even when the judgment of an individual expert or the previous “traditional” approach has been proven less effective.
James