We have announced the full results of our Predictive Analytics in the Cloud survey. The results are available as a white paper and as a recorded webinar – go to smartdatacollective.com/predictive-analytics-cloud to register for all the deliverables. There were some interesting results and I thought I would share a few.

• The core focus for predictive analytics, and for predictive analytics in the cloud, is improved targeting and development of customers. It dominated the top outcomes from predictive analytics as well as the top areas of focus in both predictive analytic sand cloud.
• All five of the scenarios – cloud-based predictive analytic solutions (decisions as a service), cloud-based deployment of predictive analytics into SaaS applications, cloud based deployment of predictive analytics to on-premise applications, using cloud-based data in modeling and pushing modeling to the cloud – were seen as powerful with no obvious winner. None of them are that widely adopted yet but, as you would probably expect, pre-packaged analytic applications did best. The runner up was the use of cloud to embed predictive analytics into on premise applications – an interesting result that shows the importance of deploying predictive analytics not just building the models.
• Decision Management was clearly an important element for successful analytic adopters. We asked companies how they used predictive analytics and overall people were split between predictive analytics providing occasional insight and predictive analytics being tightly integrated in operational systems (the basis of Decision Management). But when you focus in on those who have already seen significant positive results from predictive analytics the percentage tightly integrating predictive analytics into operations rose while occasional use dropped. Among those transformed by predictive analytics a whopping 2/3 tightly integrate their predictive analytics with day to day operations! The power of decision management.
• These more successful companies also valued different types of data for building models. Near real-time and real-time data were seen as more important by the respondents overall but among those with more experience both batch and static data scored much higher – experience clearly shows that less volatile data can be valuable too.
• Finally a couple of surprising negative results. I really thought that more experience with predictive analytics would make people more tolerant of “black box” models but in fact the percentage who really wanted transparency in their models started high (well over half) and climbed to 80% among those with the most positive results so far.
• Even success does not make people comfortable with black box models it seems. On the cloud front I really thought that transaction based pricing – pay as you go – would be a big driver but it did poorly across the board. Reducing the demands on IT and empowering the business were what people were looking for from cloud. I think transaction pricing has a lot to offer folks with decisions as a service cloud-based solutions in particular but it’s not apparent that the survey takers agree with me.

Register at SmartData Collective for more. Thanks to Clario Analytics, FICO, Opera Solutions, Predixion Software, SAS, Teradata and Toovio for sponsoring.

Cross-posted from International Institute for Analytics

There was a great article recently in the McKinsey Quarterly – The Second Economy. In it W Brian Arthur discusses the fact that

Digitization is creating a second economy that’s vast, automatic, and invisible—thereby bringing the biggest change since the Industrial Revolution.

It’s a good article and worth a read. Brian talks about the systems that drive this economy – that automate decisions so that systems can communicate and collaborate without human intervention. These systems for automating decisions have three characteristics it seems to me:

• They are very agile or “constantly changing” as Brian says – easy to change as needs and circumstances change.
  The digital economy moves faster than the physical one as it has fewer constraints on change – it’s much easier to reconfigure something electronic than it is to reconfigure something physical
• They are analytic – using the data available in the network to decide what will work best
  We have more data than ever before both inside our organizations and in the network as a whole. The systems for the digital economy consume this data analytically, using it to behave in more effective, more profitable ways
• They are adaptive or self-configuring as Brian puts it – testing and learning to see what might work better over time
  Whether people conduct the experiments or the systems conduct experiments automatically, the systems for the digital economy learn and evolve constantly to maximize their value.

These, of course, are the characteristics of Decision Management Systems. To participate in the digital economy your organization needs these kinds of systems so learn to build them.

Decision Management Systems are the next generation of information systems. Agile, analytic and adaptive, Decision Management Systems put business rules management systems, predictive analytics and optimization to work improving the effectiveness and efficiency of your operations. Decision Management Systems are agile – easy to change, easy to keep compliant and transparent in their behavior. They are analytic, using historical data to predict risk, fraud and opportunity. And they are adaptive, responding to changing market needs and supporting experimentation and continuous improvement. This series of webinars will show you how to build this exciting class of system.

How to Build Decision Management Systems: Part 1 Decision Discovery
Wednesday September 21, 10 am PT

This webinar will introduce Decision Management Solutions proven approach to decision discovery. Based on multiple client engagements, this approach identifies the decisions that matter in your business. Mapping decisions to your organization, objectives and existing systems and processes puts the decisions in context. Modeling decision dependency and capturing essential analysis information ensures you will focus on the right decisions and the right technologies. Register here.

Sometimes people talk about the value of data mining or predictive analytic modeling coming from “aha moments”, where the analytics deliver some piece of dramatic insight that enables a company to see some fantastic new market opportunity or fundamentally change the way it does something. This is only a small part of the story.

The companies I work with or know of get tremendous value from analytics by applying analytics to improving operational decisions. They use analytics:

• To better predict the fraud risk of a claim so they can pay fewer fraudulent claims.
• To better predict credit risk so they can manage credit lines more effectively.
• To predict which cross-sell offer will be most likely to succeed.
• To predict which price with convert a visitor.

These companies have an implementation process to make sure analytically derived predictions are turned into useful actions, often by applying business rules technology.

So when you think about analytics, don’t think only about aha moments, think about the operational, transactional, micro-decisions that drive your business. Think about how analytics could make each of those decisions a little better, adding up to a whole lot of value. It’s not about “aha” moments - It’s about making better operational decisions.

Decisions are where the business, analytics and IT all come together.

[Article first published in the Decision Management Solutions December 2010 newsletter]

Analytics can transform your business. I shared with you how to transform customer retention and marketing with analytics. This time I’d like to share with you how to use the power of analytics and Decision Management to allocate constrained resources and to give you a competitive edge.

How to maximize resource effectiveness with analytics

Organizations are under constant pressure to improve the efficiency and effectiveness of their operations. Budget and headcount restrictions are common place. In this environment you must focus on maximizing the resources you have by rethinking traditional process and workflows to be more efficient. What are currently complicated and resource-heavy processes are often in actuality simple processes that have been over complicated by poor decision management. You can streamline processes by finding the decisions where many manual steps are required to process data.

There are three steps to transforming resource allocation.

Step One: Decision-driven Data Integration

The backbone of analytically based decision making is integrating the data required to make a particular decision. Many organizations have data scattered across multiple sources and implement processes to manually integrate and aggregate this data. Much of this data has only ever been considered from a financial perspective. Resources are directly wasted on these processes and indirectly wasted because decisions based on assumptions and habit rather than data.

Decision Management Edge: Begin with the decision instead of the data. Many data integration efforts flounder because they attempt to integrated data without a purpose.

Step Two: Focus on High Value Decisions, automate the rest

With the data integrated, the opportunities for cost reduction abound. Business rule management systems can streamline basics like alerts that align to business objectives. But moreover, they can act automatically on some or all of their transactions using this information. Business rules can easily express the regulations and policies that need to be applied, allowing 80% or more, sometimes up to 95%, of transactions to be automated. Resources can now focus on high value decisions and have the time they need to use their data to make better ones. The combination of business rules-based automation of day-to-day transactions and analytics to minimize the exceptions to those rules, streamlines processes and the resources needed. Business rules automation makes it easier to apply resources and analytics where they can be most effective.

Decision Management Edge:  Organizations that consider business rules and analytics as a pair automate more routine manual decision making and focus human decision-makers where they make the most difference.

Step Three: Proactive value creation

Instead of waiting for something to go wrong and then spending money and resources to fix it, organizations can identify much cheaper preventative steps. They can apply resources to the prevention of problems not their cure. Predictions can be of fraud, of natural disasters or other problems, of equipment failure and much more. A rich, integrated history of what has happened in the past can provide the data needed to produce these kinds of models. Predictive modeling techniques use this data to turn uncertainty about the future into usable probabilities. Instead of being taken by surprise, because they are unable to predict when a problem will occur, organizations can estimate how likely each kind of problem is to occur in time to act. By analytically deriving where risk truly exists, organizations can focus resources where they will create the most value.

Decision Management Edge:  Using predictive analytics to detect problematic trends and estimate future risk avoids pay-and-chase fraud investigations and allows resources to be assigned to the next best case, maximizing the value of your resources.

Each step builds value – making data-driven decisions improves efficiency, streamlining processes by automating decisions frees up resources for higher value work and using predictive analytics lets you assign resources where they will make the most difference. Companies and government agencies are already using these approaches to transform resource allocation and drive value to their bottom line. This is not a future vision but a practical way to make improvement today. And Decision Management gives you the edge in quicker realized value of analytics implementations.

These insights come from the collective wisdom of over 50 organizations that have transformed their business using analytics. These organizations tackled core business issues like customer retention, marketing, service delivery, operations and customer centricity, in industries as varied as Telecommunications, Retail, Healthcare, Education, Government and Banking.

Analytics can transform your business. Last time I shared with you how to transform your marketing with analytics. This time I’d like to share with you the power of analytics to optimize customer retention and how to use a Decision Management approach to give you a competitive edge.

How to transform customer retention with analytics

Conventional wisdom is that the cost of retaining an existing customer is far less than acquiring a new one. As a result, customer attrition rates are a key business performance metric in many industries from telecommunications to financial services, online retailing, Pay-TV and more. But a focus on attrition or retention rates is only half the story, you also need to focus on the cost of retention. Optimal customer retention is about retaining the right customers at the right price. And this means knowing who the right customers are as well as what will retain them most cost-effectively.

There are three steps to transforming customer retention with analytics:

Step One: Build customer understanding

Most companies have two main barriers to building customer understanding—siloed data and competing business units. Data is not organized around customers, and different channels and business units keep their data separate making analysis impossible. Business units and product lines compete for customer attention resulting in product-centric rather than customer-centric offers. Each business unit makes its own offers with little or no knowledge of the overall impact on the company.

Decision Management Edge: Companies that created and used a single view of the customer across channels and business units were able to quickly develop fine–grained segmentation and to monitor customer behavior so they could rapidly respond to developing trends.

Step Two: Establish differentiated treatments

Most companies segment their customers only by product type, by region or by original channel. When retention offers are made they are largely undifferentiated–the same for everyone. More is needed to enable true differentiation. Data mining techniques can find groups of customers who have a desired characteristic—such as a likely positive response to an offer—as well as groups who are generally similar. These descriptive analytic techniques deliver cluster or segment definitions that divide customers into meaningful groups. Combined with an infrastructure for delivering different offers based on these segments, these techniques can enable truly differentiated treatments.

Decision Management Edge:  While analytic segmentation adds to a company’s understanding of its customers, it does not automatically deliver differentiation. Companies need to ensure that the analytics drive differentiated decisions at every customer contact. The effectiveness of these decisions—what offer to make to a customer threatening to leave or which customers to proactively target—drives customer retention.

Step Three: Focus on predictive decisions

Step two used historical data to understand customers. Companies can take this one step further and use this historical data to look into the future. Predictive analytic techniques take uncertainty about the future and turn it into a usable probability. Instead of being uncertain which customers are churn risks, companies can assess the probability that each customer is such a risk. Historical data about the behavior of customers, the profitability of products over time, service cancellations and more can be used to calculate the probability of a behavior for a given customer. These techniques can estimate the likelihood that a specific customer will churn in the next 30 days, for example, or predict which offer they will find most compelling. With these probabilities in hand, customer treatment and retention decisions can be made more accurately, creating offers that are more likely to be accepted and targeting those customers at greatest risk.

Decision Management Edge:  Companies who developed predictive models to find their most desirable and at-risk customers and used those models to improve decisions at every touch point were able to focus retention efforts and uncover new paths to increased revenue.

By building customer understanding, establishing differentiated treatments, and focusing on predictive decisions, companies like yours can make better customer retention decisions. Every decision is differentiated, based on what has worked in the past as well as what is likely to work in the future. Every decision is based on the customer and on what worked with similar customers. Every decision focuses on retaining the right customers at the right price.

Each step builds value – using integrated customer data improves programs at the macro level, differentiating treatment makes your offers more compelling, adopting predictive analytics target based on future value and future risk. Companies are already using these approaches to transform their retention strategies and drive value to their bottom line. This is not a future vision but a practical way to make improvement today. And Decision Management gives you the edge in quicker realized value of analytics implementations.

These insights come from my research and are based on the collective wisdom of over 50 companies that have transformed their business using analytics. These companies tackled core business issues like customer retention, marketing, service delivery, operations and customer centricity, in industries as varied as Telecommunications, Retail, Healthcare, Education, Government and Banking.

How to transform your marketing with analytics

Companies need to develop a loyal customer base and market effectively to them. But customers increasingly shop in multiple channels and formats, and have more choices than ever before. Your company can leverage the power of analytics to overcome these challenges. You can develop the capability to target customers with personalized marketing, building brand loyalty and increasing sales. What’s more, each step of this transformation —integrating disparate data sources to give a complete picture, creating a customer-centric perspective, micro-segmentation and ultimately personalizing marketing—creates value and improves outcomes.

Step One: Integrate Information

Most companies I work with still keep information in multiple databases with different regions or different divisions storing their own sales data. Product data is managed separately. To build a customer perspective that is actionable, step one is to map your sales to your product data, with analytics in mind. This means sales on a store-by-store, channel-by-channel basis at a granularity that allows for the analysis of sales by time of day or for “the week before Thanksgiving” and product data that is normalized across channels and captured to the package level.

Decision Management Edge:  Begin with the Decision in Mind. Companies that collected and integrated the data they needed for a specific decision like ”is this product out of stock” had shorter implementation times than those with a more general approach.

Step Two: Build a customer perspective

Once a solid base of information exists about products and sales, the next step is to match it to customers. The goal is to tie all sales data to specific customers to create a customer-centric information set. Companies can then begin to see the kinds of customers they have and use this understanding to enhance their marketing campaigns, store layouts and more.

Decision Management Edge:  Integrating the data for the decisions that make a difference to customer loyalty and profit realizes value more quickly than just aiming for a 360 degree view of a customer. Decision Discovery Services find and prioritize your customer decisions.

Step Three: Develop micro-segments

With customer sales and product data mapped to specific customers, you are ready to build analytical models. With your rich set of customer-centric information, you can now apply data mining and micro-segmentation. These analytical models will predict future customer behavior and develop fine grained segmentation or micro-segments. For example, predictive models can assess the likelihood that a particular customer will respond to a particular kind of offer or use the web channel.

Decision Management Edge:  Companies that incorporated decisioning technologies like business rules in their front line IT applications achieved a higher ROI on their analytics investment. These companies are able to deliver analytic differentiation to their call center, their website and every customer interaction. Develop a Decisioning Technology Blueprint to operationalize your analytics.

Step Four: 1:1 marketing

The analytical models you have developed using your rich data set now enable truly 1:1 marketing. Companies who have reached step four are recreating the corner store using analytics. They are taking all the information they already have about all their customers, analyzing it and applying it so they can get to know and understand each individual customer. In the same way that the owner of a corner store knows every customer, their preferences and their needs, a large company can use analytics to deliver the same sense of connection at scale: a corner store with thousands, hundreds of thousands or even millions of customers.

Decision Management Edge:  It’s now easier to get smarter and smarter about the decisions that matter to your customers. Decisioning technology like business rules makes it easy to evolve and change decision making without impacting the rest of your systems. Add predictive analytic models to support proactive decision-making and experiment with multiple approaches to find the optimal decisions for each “market of one”.

Each step along the way to transforming your business through analytics builds value. Companies at Step One can now do comparisons and planning across channels, geographies and store formats. At Step Two, with a solid customer centric information platform, companies can identify and link all information around a common definition of a customer. At Step Three, with customer information in hand, companies can create customer personas and increasingly fine-grained segmentation that allows more precise targeting of marketing. At Step Four, leading edge companies are using advanced analytics to recreate the corner store, delivering tailored offers and incentives that are personalized to each customer based on their past behavior and their likely future behavior.

Transforming your marketing with analytics is not some far off future vision. Companies today are crafting personalized, targeted offers at the point of sale and designing new customer-centric campaigns that increase revenue. Decision Management can be your edge in more quickly realizing the value of analytics.

I want to share with you the collective wisdom of over 50 companies that have transformed their business using analytics. These companies tackled core business issues like customer retention, marketing, patient care, student achievement, and customer centricity in industries as varied as Retail, Telecommunications, Healthcare, Education, Government, Insurance and Banking.

Tomorrow I will kick off a three part series on the blog with how you can transform your marketing with analytics and how Decision Management gives you a competitive edge. The series is based on a set of newsletters I published a while ago and will cover

• How to transform marketing with analytics
• How to transform customer retention with analytics
• How to maximize resource effectiveness with analytics.

I look forward to your comments and feedback on the series.

IBM recently surveyed CIOs as part of their ongoing CxO research. This was the second time they did this – 2009 was the first. They just released the results of their analysis of the 3,000 interviews they conducted in 71 countries. The results are summarized in the body of the post along with some Decision Management advice related to it.

CEOs have said their focus for the next 5 years are Getting closer to the customer, People skills, and Insight and intelligence. Meanwhile their CIOs focus on Insight and Intelligence, Client intimacy, and People skills (it’s a pity CIOs don’t think they can help with “getting closer to the customer” as I believe the effective use of analytics can do just that). CIOs see their most important “vision” elements as Business Intelligence and Analytics (stayed top from 2009 study), Mobility (up somewhat), Virtualization (down somewhat), Cloud (way up), Business Progress Management (down slightly) and Risk Management and compliance (down quite a bit).

IBM identified 4 categories of CIO Mandates and strongly recommends that organizations work across business and IT organizational boundaries to decide what the right CIO mandate is for their organization. This might change over time but each kind of Mandate has best practices that can only really be applied if everyone is explicit about it. IBM also asked questions to determine which IT organizations “outperformed” their peers so that they could contrast outperformers with others. The four CIO Mandates are:

• Leverage – streamline operations and improve effectiveness
• Expand – refine business processes and enhance collaboration
• Transform – change industry value chain through improved relationships
• Pioneer – radically innovate products, markets, business models

Let’s consider each in turn to see their differences and areas of focus according to IBM as well as both IBM and my advice for these CIOs.

Leverage CIOs

Leverage CIOs are focused on IT fundamentals and the business primarily sees them as a provider of basic technology services. They need to continually improve their legacy systems, improve their sharing of data with internal customers and use data to measure basic business and IT metrics. Key elements include rationalizing their application portfolio, gradually altering their hardware and, for some renew their IT environment. Many of these CIOs are in organizations with lots of assets and complex supply chains –these companies have the majority of their assets tied up outside of IT.

CIOs with a Leverage Mandate in outperforming IT organizations tend to do better with sharing information with internal clients and have more people who can cross over between business and IT. To excel, IBM recommends 5 things:

• Focus on business/IT communication
• Build the right team for each task
• Standardize and consolidate infrastructure
• Update and renew legacy
• Discover dashboards for business and IT metrics

Decision Management advice for Leverage CIOs? Discover Business Rules Management Systems as a way to streamline and improve legacy applications (see this blog post on Decision Management and legacy modernization for more).

Expand Mandate

Expand CIOs are focused on optimization of technology to improve integration and effectiveness. The business sees them as a provider of technology, a facilitator of efficiency and a provider of industry specific solutions. Four key things for these CIOs are exploring outsourcing, enhancing internal collaboration, implement Business Process Management and continually update their legacy environment. They are aggressively changing the kind of skills they have in house and are partnering/outsourcing extensively. This is about 50% of the CIO population.

CIOs with an Expand mandate in outperforming organizations are much better at internal collaboration and have a ruthless focus on integration of business and IT. IBM recommends 5 things:

• Enable state of the art collaboration
• Increase business/IT integration and transparency
• Focus on core and involve partners for everything else
• Simplify, automate and integrate with BPM
• Provide more sophisticated dashboards

Decision Management advice for Expand CIOs? Use Business Rules Management Systems to improve transparency and business/IT collaboration (see this white paper also on how to maximize the value of business rules with Decision Management) and focus on Decisions as well as Process to built smarter, simpler more agile business processes (see this white paper on the decisions at the heart of your process).

Transform Mandate

Transform CIOs are not focused on fundamental provision of services but focus on industry specific solutions as well as efficiency and enabling the vision. These CIOs have a lot of data that they want to use, they need to simplify things and expand the value chain outside the organization while increasing business/IT intimacy. Transform CIOs are using data to create insight and drive better real-time decisions primarily with client analytics (analytics about customers).

Outperforming Transform CIOs are better at driving better real-time decisions than others and are simplifying how external partners can work with them. IBM recommends 5 things

• Simplify and remove unnecessary complexity
• Real time dashboards
• Extend the value chain outside the organization
• Use more real-time data
• Dive into advanced analytics

Decision Management advice for Transform CIOs. Remember that advanced analytics will add no value unless you can put those analytics to work in your systems so use Decision Management to make sure you can deploy and use your predictive analytics (see this white paper on Putting Predictive Analytics to Work in Operations). Real-time decision-making means having systems that make decisions, not just people that do. Remember also that using BPM for everything can result in over-complex business processes so make sure you focus also on Decisions to simplify those processes (see this white paper on the Decisions at the heart of your Process).

Pioneer Mandate

Pioneer CIOs are perceived as being a critical enable of the organization’s vision while still providing industry specific solutions. They focus on using predictive analytics to change the business, search for new sources of revenue and on taking customer experience to the next level. These CIOs are mostly in information-centric industries but are a minority even there. These CIOs do a lot of work on how money can be made in the future – revenue model changes, new sources of revenue, future profitability predictions etc.

Outperforming Pioneers focus more on profitability analytics and on social network analytics. IBM recommends 5 things:

• Innovate on the top line
• Act on deep customer understanding
• Exceed expectations
• Develop an analytic culture
• Add predictive dials to your dashboards

Decision Management advice for Pioneer CIOs – go big on Decision Management. Improving customer treatment means taking (analytic) control of customer treatment decisions (see this post for more on customer treatment management with Decision Management) and to make the most of predictive analytics you must be able to put your predictive analytics to work in operational systems (see this white paper on Putting Predictive Analytics to Work in Operations). Finally add knobs to your dashboard so it becomes a cockpit – make it possible for dashboard users to change the behavior of the systems behind the dashboard not just look at it.

You can get the report and more at http://www.ibm.com/theessentialcio

I got an invite to a webinar on this topic the other day. The invite had some questions for you to ask yourself about your call center agents and how effective they would be if:

• They could act the way you wanted them to every time
• They didn’t have to have post-its or cheat sheets
• They didn’t need to spend months in training
• They could act like your best agent?

These are the right questions, especially for an organization with a reasonable number of call center agents. Part of the answer lies in being able to implement consistent processes for your agents to follow and deliver a usable system (these were the key elements of the solution being discussed in the webinar). Part of it lies in being able to present the right information to agents as they are working to address customer issues. What’s missing in many call centers, however, is a system that helps them make the right customer treatment decisions.

After all call center agents must decide how to act, how to treat customers.They must act in a way that is legal and complies with company policy. Often the rules implied by regulations and policies must be enforced manually with policy manuals and training. You are reliant on the agent themselves remembering and applying all the rules. Relying on this manual approach makes for errors and inconsistency – many have had the experience of getting an answer they did not like from a call center and then calling back to complain and getting a completely different one. In contrast to this manual approach, embedding the rules in a Decision Management system would allow the system to guarantee compliance and ensure consistency. It could (and should) even define when exceptions can be made and ensure that agents who make exceptions record their rationale.

With this kind of system in place the need to keep cheat sheets handy or put rules on post-it notes goes away – exceptions and changes can be made directly to the rules in the system quickly and effectively. I did some work in a call center that was implementing a Business Rules Management System so it could build this kind of environment and we found many cheat sheets that described how to pull data from various systems and then use that data to make a decision. Once the decision was automated the need for the cheat sheet would go away.

Of course this kind of system also requires far less training. If the system moves from passively supporting decision-making to actively managing it then the user needs far less training in the nitty gritty of policies and other rules. Decision Management systems are also more active, more likely to act appropriately on behalf of the customer or agent, and this often simplifies the user interface (further reducing training). In the previous example, for instance, the agents no longer need to be taught how to extract data from various systems as this is handled under the covers by the decision service that implements the decision.

Finally these kinds of system actually enable all your agents, on their first day, to act the way your best agent would. Capturing best practices and tribal knowledge as business rules is an established approach and a Decision Management system let’s you embed those kinds of rules as well as your policies and regulations. In fact it would let you go further as it supports the embedding of predictive analytics (which have a good track record of out performing human experts) and the management of ongoing test-and-learn experiments to see what works best and should become the new best practice.

So, if you really want to make sure your agents can handle any call, implement some Decision Management systems to help them.

Here are some other posts on this topic you might enjoy

• Using Decision Management to power the call center of the future
• Analytics in the call center
• Using decision management to improve first call resolution
• Using Decision Management to manage call center and other costs
• Some thoughts on using analytics about your staff to improve customer treatment
• Investing in your lowest level employees with Decision Management