Nina Shikaloff discussed an analytics technique that I had not heard of – Impact Modeling. Impact Modeling is a decision modeling technique. Decisions on acquiring customers – what to offer for instance – managing customers and handling difficult customers are all important and it can be tricky to identify better ones. Impact modeling is about explicitly measuring the incremental financial impact of a strategy – how much more will I make if I do ‘B’ rather than ‘A’ – and then mapping segments of customers to the optimal decision.
However you can’t test any particular customer with both strategies to compare results so instead you use adaptive control (A/B testing) to try A on 90% (say) and B on the other 10% while tracking the results. The Impact Modeling algorithm then searches through the results to see which segments respond better to which strategies. Essentially it uses the results to find segments where one particular strategy works better and keeps driving down into the details of these segments to find more and more fine-grained ones where one approach or the other works better. The outcome is a decision tree or a simple ruleset that picks one of the strategies for each segment – very deployable. It is also easy to simulate the impact of the approach allowing you to maximize the financial impact.
Impact Modeling can be used when tracking multiple financial objectives and can be constrained by competing objectives (risk v revenue, for instance). It can also be extended to more than 2 choices and can be used on relatively small samples.
Nina illustrated the power of Impact Modeling with a couple of case studies. The first was a credit card issuer trying to find the right APR increase that would boost revenue without increasing risk or attrition. They found that half the accounts should get an APR increase (some small, some larger) while the other half should not to maximize results. Each strategy was applied to multiple segments and one of the interesting effects of Impact Modeling is this understanding of the segments. The second was another credit card issuer with a very diverse target group and learning which sub-segments responded to the two offers was very informative. Not only did Impact Modeling get better results, the user learned a lot too.
Given the outcome is a decision tree it may seem like Impact Modeling is the same as normal decision tree modeling. Impact Modeling is essentially an analytic technique for finding the right rules because it analytically finds the right tree nodes, considers the impact of prior decisions and allows multiple objectives to be considered. Personally I really like these kinds of analytic techniques as they are so clean to deploy, allowing them to be put into production rapidly. This is something that I will touch one when I speak this afternoon.
