IBM announced an interesting new analytic product today – RAMP or Real-time Analytic Matching Platform. The idea behind this is to improve on the approaches currently used to route calls which typically rely on an availability-based approach to connect customers to agents. This doesn’t take an agent’s focus or availability nor what a customer is interested in.

Instead of being put into a queue and connected FIFO style, this new product uses analytics. The system analyzes you as a customer, up to and including what you say you are interested in. They summarize this with a score to see how good a customer you are and then do the same thing on the agents – how good is this agent at solving the issue you have. Analytics match you to an agent in real time considering these scores, availability/wait time v taking you to someone less optimal etc.

Surprisingly enough IBM finds that companies do have the data they need for these analytics, albeit stored in lots of different sources. The system also supports explicit rules to constrain choices but they prefer to “let the data speak” having found that call centers are often very dynamic with rapid changes in staffing/skills/effectiveness getting out of synch with the rules. They do use rules to handle things like SLAs for wait time etc. A dashboard also allows managers to “move levers for overflow/short agent staffing”.

I find this product fascinating. Those of you who have heard me speak no that dumb IVR/call center systems are one of my pet peeves and the idea of using analytics in this way is great. Interestingly this is a “matching platform” and could be used not just for agents but also to drive an IVR for example.

IBM says it has successfully deployed this new approach at Assurant. Assurant is a specialized provider of insurance products that says:

Assurant Solutions has been using an analytics-based routing approach to increase call center profitability and enhance the customer experience in its call centers for more than seven years, increasing retention revenue by 37% and sales revenue by 29% within the first year of implementation

IBM has posted an animation that explains how RAMP works on You Tube http://www.youtube.com/watch?v=ibL2rpNHXmo.

Got a quick update today from the new co-CEOs of SAP – Bill McDermott and Jim Hagemann Snabe.

Jim focused on their innovation strategy – making significant steps into on-demand business applications, aiming to support a hybrid approach allowing customers to mix on-demand and on-premise software. In addition they aim to increase support for running the applications on new mobile devices – this, of course, requires a separation of decision-making business logic from front-end logic. Hopefully this will see SAP investing more in its business rules capabilities (described under the SAP tag on the blog). All of this requires that processes and MDM can be orchestrated across this increasingly complex environment, even when non-SAP application components are involved. They are also rolling out more agile development methodologies (like those being described in the new SAP BPM book on which I am working with various other SAP folks).

Lots of interesting questions got asked and here are some of the responses that seem most interesting from a decisioning perspective:

• In memory analytics will change the way high end analytics are deployed. Focused on a variety of partners to bring new approaches, new techniques into high-end analytic space. Still expect to work with SPSS in this regard but also looking for new technologies that take advantage from the ground up of in-memory analytics.
• Business by Design, SAP’s easy to configure on-demand (SaaS) offering, is coming out this summer. Will be interesting to see the extent to which business rules are used to make it configurable.
• Interesting challenge for a company like SAP is that different product lines, different deployment options have a different cycle. On premise software, for instance, cannot be updated too often as customers don’t want to constantly re-install. On-demand software, however, gets updated more often and on-device software is driven by a very dynamic consumer technology market. This is a large scale change, ensuring that different parts of the company can operate on the right timescale while remaining part of the same company. Personally I think that rules-based decision evolution is a key element of this and I hope to see some sign that SAP thinks this way too.
• Asked about mergers and acquisitions – the point was made that Oracle has been much more aggressive – Bill and Jim acknowledged that they are going to be more aggressive going forward while remaining focused on innovation and an integrated, coherent business application suite rather than generating growth through acquisitions. As more and more established customers have been acquired (up to the acquisition of Chordiant by Pegasystems today) this is an interesting topic – increasingly SAP will have no option to grow through acquisitions but this may suit their corporate culture better anyway.
• Asked about the trend (Oracle, IBM) to mix hardware and software they replied that they see a heterogeneous world that is in constant flux – customers never own one vendor’s complete set – so being good at working in this environment is key. Eliminating layers using hardware is good but they see working with multiple partners not owning their own. This requires collaboration with a mix of hardware partners rather than acquiring and integrating their own hardware. Customers don’t want vendor lock-in, they buy a business outcome not a “stack”.
• SAP is not worried about the ownership of Java by Oracle – they see a vibrant, open, multi-company ecosystem around Java and don’t expect Oracle’s ownership to impact this. Interestingly they made the point that programming languages come and go and that Java is not therefore the be-all and end-all.

Interesting conversation, nice degree of openness and responsiveness – much improved over SAP TechEd where avoiding questions was the order of the day.

The news today is that Pegasystems (rules-based business process management) is acquiring Chordiant (decision-centric CRM). This is interesting news as it merges a company (Chordiant) with a very decision-centric/decision services separate from process mindset with one (Pega) that has mixed rules and process together much more.

Chordiant have been one of my companies to watch for a while, with a great decisioning platform. Their clear separation of decisioning, their support for rules and analytics in combination, their strong adaptive analytics engine for self-learning models, their recent integration of real-time conversations with decision management and their powerful business simulation tool (Visual Business Director, see below) are enough to put them at or very near the top of the decisioning vendors.

Pega, of course, have been best known for their business process management focus. They have always addressed this from a rules-centric perspective and we have had some active disagreements about the role of decision services and the value of a clear separate of processes and decisions (see this post and comment thread, for instance). Nevertheless we agree on the power of business rules to drive more agile and smarter systems and Pega has been one of the rules vendors active in supporting PMML (Predictive Model Markup Language) to allow the integration of business analytics with business rules.

The potential of this merger is real. Clearly the merged company will be larger, important as the big players (IBM, SAP especially) get more serious and rules and decisioning. Chordiant’s decision management and simulation components are, in my estimation, better than Pega’s for specific purposes but not as general purpose. An intelligent combination of the two is therefore potentially very powerful. In particular, bringing Chordiant’s adaptive analytics and simualtion capabilities to the broader rules-based platform that Pega offers could be great. In addition both are very focused on CRM or at least on customer treatment decisioning, and this should help give the merged company a clear focus.

The risk, of course, is that the fairly serious difference of perspective between decision-centric / decision management on one hand and rules-driven BPM on the other will derail the technical integration or cause the merged company to merge its operations without merging its products. Either will ensure that the talented people behind the products will not stay and that would be a pity. The merged company must figure this out and make some clear statements on product direction and positioning in this respect – though I appreciate that this can’t be done right now, as it must wait for regulatory clearance etc. There is a lot of overlap in technology. This could be good – giving the merged company enough of a common vocabulary to build a powerful solution – or bad, resulting in lots of infighting about which version to keep.

Check out these posts on Chordiant for more details. The folks at Pega have never seemed to want me to blog about them so I don’t have anything about them on the blog. Hopefully this will change…

• First Look – Chordiant Decision Management
• First Look – Chordiant Recommendation Advisor
• First Look – Chordiant’s Visual Business Director
• Chordiant Decision Management Update

Disclosure: Chordiant was a customer of mine in 2008/2009

Syndicated from BeyeNetwork

In a great post on 8 things to keep in mind on predictive analytics, some folks from Diamond Management & Technology laid out some things to keep in mind that I really liked. Here they are with my comments – you can get more detail on each from the series of posts with which they followed this initial one.

1. Understanding the cost of a wrong decision helps target investments
   Absolutely, though I still think that finding a decision you can tie to an executive’s compensation plan works better.
2. Strategic and operational decisions need different predictive
   modeling tools and analysis approaches
   .. and deployment approaches. I divide decisions into strategic or direction-setting ones, tactical or day-to-day management ones and operational or transactional ones. Particularly with the latter, which are crucial, you need to think about how the models will be deployed if they are to add value.
3. Integration of multiple data sources, especially third-party data,
   provides better predictions
   Yup, but don’t just integrate your data – begin with the decision in mind and integrate to support it.
4. Since statistical techniques and tools are mature, by themselves
   they are not likely to provide significant competitive advantage
   True. It is their ability to turn YOUR data into YOUR insight that does.
5. Good data visualization leads to smarter decisions
   .. at the strategic and tactical level and to better models at the operational level – decision making at the operational level is too high-speed, too automated for much in the way of visualization to be useful a the moment of decision.
6. Delivering the prediction at the point of decision is critical
   Yes!
7. Prototype, Pilot, Scale
   Of course – don’t forget to scale the deployment piece too
8. Create a predictive modeling process & architecture
   Yes. And map it to your IT development process if you want to impact operational decisions embedded in your enterprise IT infrastructure.

A great list!

I just heard from a colleague that you can check out Oracle’s Data Mining tools on the amazon.com compute cloud.  The Oracle Data Mining development team has set up an instance for prospective customers who want to try the in-database data mining algorithms via SQL/Java APIs or the Oracle Data Miner user interface. You can launch an Oracle Data Mining Amazon Machine Image (AMI) directly through Amazon Web Services (AWS) and your only cost is the standard Amazon EC2 charges.

To get started go to the Amazon Cloud with Oracle Data Mining or click here for a step-by-step visual guide. There’s more on the Oracle Data Mining blog – http://blogs.oracle.com/datamining/

I was traveling in South Africa last week (keynoting BI 2010) and my favorite online payment system demonstrated not once but twice, why business rules are so valuable in analytic decision making. First their analytics triggered a fraud alert – presumably based on patterns of problems from South African IP addresses. As I was trying to make a payment to a vendor I have used before, a simple rule could and should have overridden this (if the payment is to someone who has been paid before without problems then it is not likely to suddenly become fraudulent). But it did not and I had to go through the unblock account process.

Sadly the unblock account process assumed you were at home. Of course, I was not, and there was no way to unblock the account. This is a problem as most fraud alerts are false positives (that’s just how it works) so the assumption that I am traveling would have been more reasonable than that I was at home. Some rule to say that, if it was me and not a fraudster, that I was likely to be away from home and so could not answer my home phone would have been helpful. As it was I had to leave the account suspended until I got home.

The account is now unblocked but, trying to be helpful, I sent them a note explaining the situation. Sadly their use of text analytics was also rules-free. Picking up the words in my email about suspended account processes it simply sent me a note about how to unblock my account. Again, a simple rule would have discovered that I had, in fact, already unblocked my account and so probably did not need help on that topic. This would have allowed them to route it to someone who could have answered my email instead of merely irritating me with an automated response.

Analytics can detect fraud and give you a good sense of what an email says. But explicit business rules can map that insight to current business conditions and make sure you action makes sense. Use them together and make better decisions.

Syndicated from BeyeNetwork

My co-author on Smart (Enough) Systems, Neil Raden, has written a great white paper on in-database analytics that is available from Sybase – Analytics from the start. This paper introduces the key concepts, discusses some of the key issues (our book contains more tips in this area) and describes some strong case examples. Well worth a read. As Neil says:

Advanced analytics will be adopted by most organizations and attain the status of “must have.” While the majority of people in organizations will not become quantitative experts and modelers, the affect of predictive models will be felt across the organization and beyond. They already are. It would be wise to take steps now, and a good first step is to begin evaluating technology solutions that will be suitable for the development and implementation of analytics. From a technology perspective, one clear requirement is an analytic engine embedded in your analytical database technology.

The approach Neil describes is one we see more and more as in-database and
in-warehouse analytics become more common. This particular paper talks about the Fuzzy Logix libraries embedded in Sybase IQ but . Fuzzy Logix is one of the sponsors (with SAS, Oracle, Adaptive and Aha!) of the operational analytics research I am doing for BeyeNetwork. Look for it on the BeyeResearch site in a couple of months and, meanwhile, participate by taking the survey.

Doug Henschen has a great interview with Deepak Advani of IBM, the new head of IBM’s newly acquired SPSS business (and I am not just saying that because he mentions Smart (Enough) Systems).  I am looking forward to seeing what IBM does with the combination of ILOG and SPSS, along with InfoSphere, WebSphere, FileNet and Cognos.

Eugene from SARS, the South African Revenue Service, presented next on how SARS is using BI in revenue collection. He began by pointing out that there is a difference in how public sector organizations use BI – a focus on service delivery not profits, on taxpayers not customers, enforcement campaigns not marketing campaigns and so on. Of course public sector organizations still want an ROI, operational efficiency and use KPIs for performance management.

SARS has a wide range of core systems as well as a set of external data sources. Initially the IT department just dumped data from the source systems to their business users. This was replaced with a more formal information management department that responded to requirements defined by analysis teams but still hit the source systems. Capacity constraints led to an enterprise data warehouse (Teradata) but the Information Management department could not meet the demand for new reports etc while the business users wanted more control. Their current state is that of having their information management department acting as an enabler for business departments to manage their own BI capabilities. The technical architecture behind this has a primary staging layer for moving data into a production warehouse Operational Data Store and a secondary staging area supporting BI and data mining warehouses. This two stage approach allows them to present historical data through the lens of constantly changing business rules. A metadata repository underpins this and a presentation layer gives users access to reports, cubes etc.

SARS presents strategic summaries, aligned with the KPIs, as dashboards for the executive level who are typically considered measurement users. Tactical reports and dashboards are delivered to regional offices. These users tend to be exploratory users. Finally operational intelligence is delivered to execution users at the operational, branch level. The different levels consume different kinds of analytics.

SARS has learnt not to pursue big bang projects, to mix business and IT people, to plan for poor data quality and for peak season volumes and to manage change. From a business perspective they focus on changing how business people request data/reports, on showing ROI and on embracing user empowerment and self-service.

They use standard reporting on things like ontime filing, with an ability to drill down into zones, industries and more as well as self-service for reporting on metrics against various dimensions, slice and dicing etc. More interestingly they use various advanced analytics to catch fraud etc. For instance, a company might under report its corporate income tax and over-report the VAT it paid so that it continually gets refunds. However, this is a challenge because:

• Some critical fields are not mandatory
• It can be hard to correlate these two kinds of tax return
• Suspicious activity may have been reported but it is purely unstructured text.
• At the end of the day the intent is to find those organizations who are truly suspicious so data on registration, status, payment rates/timeliness must also be considered.
• And not everyone can be pursued so who to call and who to audit.
• Finally, are there linked entities that need to be closed down when a fraudster is found.

Advanced analytics are used in various ways:

• Neural nets predict values, or at least buckets of values, for missing values
• Statistically infer outliers
• Text mine the unstructured text reports to see if there are patterns of reporting that will allow early investigation
• All of this feeds into a risk engine that predicts the risk of fraud
• They then predict who is likely to be reached by the call center to prioritize calls to these taxpayers
• Next they predict the likelihood of a successful audit so that the auditors can prioritize their work
• They use association and geospatial data to find clusters of suspicious organizations, linking directors, audit companies etc.
• 3rd party information is brought in on things like houses and assets, travel etc to find suspicious mismatches between tax returns and lifestyle.

Great example of advanced analytics to detect fraud and catch tax evaders.

Eugene from SARS, the South African Revenue Service, presented next on how SARS is using BI in revenue collection. He began by pointing out that there is a difference in how public sector organizations use BI – a focus on service delivery not profits, on taxpayers not customers, enforcement campaigns not marketing campaigns and so on. Of course public sector organizations still want an ROI, operational efficiency and use KPIs for performance management.

SARS has a wide range of core systems as well as a set of external data sources. Initially the IT department just dumped data from the source systems to their business users. This was replaced with a more formal information management department that responded to requirements defined by analysis teams but still hit the source systems. Capacity constraints led to an enterprise data warehouse (Teradata) but the Information Management department could not meet the demand for new reports etc while the business users wanted more control. Their current state is that of having their information management department acting as an enabler for business departments to manage their own BI capabilities. The technical architecture behind this has a primary staging layer for moving data into a production warehouse Operational Data Store and a secondary staging area supporting BI and data mining warehouses. This two stage approach allows them to present historical data through the lens of constantly changing business rules. A metadata repository underpins this and a presentation layer gives users access to reports, cubes etc.

SARS presents strategic summaries, aligned with the KPIs, as dashboards for the executive level who are typically considered measurement users. Tactical reports and dashboards are delivered to regional offices. These users tend to be exploratory users. Finally operational intelligence is delivered to execution users at the operational, branch level. The different levels consume different kinds of analytics.

SARS has learnt not to pursue big bang projects, to mix business and IT people, to plan for poor data quality and for peak season volumes and to manage change. From a business perspective they focus on changing how business people request data/reports, on showing ROI and on embracing user empowerment and self-service.

They use standard reporting on things like ontime filing, with an ability to drill down into zones, industries and more as well as self-service for reporting on metrics against various dimensions, slice and dicing etc. More interestingly they use various advanced analytics to catch fraud etc. For instance, a company might under report its corporate income tax and over-report the VAT it paid so that it continually gets refunds. However, this is a challenge because:

• Some critical fields are not mandatory
• It can be hard to correlate these two kinds of tax return
• Suspicious activity may have been reported but it is purely unstructured text.
• At the end of the day the intent is to find those organizations who are truly suspicious so data on registration, status, payment rates/timeliness must also be considered.
• And not everyone can be pursued so who to call and who to audit.
• Finally, are there linked entities that need to be closed down when a fraudster is found.

Advanced analytics are used in various ways:

• Neural nets predict values, or at least buckets of values, for missing values
• Statistically infer outliers
• Text mine the unstructured text reports to see if there are patterns of reporting that will allow early investigation
• All of this feeds into a risk engine that predicts the risk of fraud
• They then predict who is likely to be reached by the call center to prioritize calls to these taxpayers
• Next they predict the likelihood of a successful audit so that the auditors can prioritize their work
• They use association and geospatial data to find clusters of suspicious organizations, linking directors, audit companies etc.
• 3rd party information is brought in on things like houses and assets, travel etc to find suspicious mismatches between tax returns and lifestyle.

Great example of advanced analytics to detect fraud and catch tax evaders.