Jim Goodnight, CEO of SAS, and Geoffrey Moore, author of Crossing the Chasm among other books, had a discussion on the use of analytics in innovation. Several areas were touched on from the global economy to innovation approaches to education.

The growth of low cost competitors in the global economy has commoditized that which can be commoditized, said Geoffrey, so companies in developed nations have to find ways to differentiate to sustain their profit margins. How to get these higher returns is getting more complex as the old advantages, of simply having more technical skills, degrade. Instead things like an ability to try and fail, and a balance between innovation and optimization (effectiveness and efficiency, say) are essential. For instance, grid computing might reduce a job from many hours to a few minutes and that’s optimization but it also allows innovation by making it possible to try things more rapidly, cycle more rapidly.

Geoffrey made some great points about the global economy. There are clear positives in the growth of new economies and the general rise of the standard of living in those companies. This does represent a challenge though. Not, perhaps to companies that are increasingly global and able to take advantage of the world economy, but to the middle of the economy. The top of the US economy is still generating innovation and talent that is competitive worldwide. But below that is a layer that no longer has a social contract with employers and that is struggling to be competitive in a more global economy.

Innovation must have a purpose – it must create differentiation from your competitors. You need to innovate consistently enough in a particular area to create a compelling differentiation from your competitors – be different from your competitors. Ongoing investment and steady improvement may be more useful than some great leap – SAS, for instance, prides itself on having made consistent investment in analytic and software R&D over many years. One of the hardest things in innovation is picking the right place to make the ongoing investment – finding the "crown jewels" of an organization, the key strengths of competitors and the needs of customers is critical to drive this decision.

There was some discussion of the education system and the challenges of education in a global economy. Most kids who drop out of high school, and 30%+ do, report that they drop out because they are bored. The fact that educational style (teacher at the front talking about a subject) has not changed in the last 50+ years despite the huge change in the interactivity and technology usage of the new generation is contributing to this. SAS has developed curriculum material and is focused on creating a more interactive, computer-based, technology-enabled environment. Curiously neither speaker spoke about the drag on the entrepreneurial economy caused by the fragmented and inefficient US healthcare system.

The general impact of the internet in terms of disintermediating everything from IT departments to schools and its ability to support self-organizing groups is that massive change to the way organization work. Building on this, getting ahead of it, is going to be critical as it cannot be resisted, only leveraged.

Finally the scale of the economy and the fact that it means systems have to be used to manage the data, manage the customer interactions is driving analytics. If interactions are so numerous or products are so varied that computers must be used then optimizing your business, making this work for you involves analytics. Over the last 30 years the shift has been from presenting analytics to people so they could make decisions to systems where no human being can be in the loop and the analytics drive decisions. These systems require a second loop, outside the transactional environment, to manage and improve the analytics in the system. This is essential because it is only the people that have a strategic intent that can drive the system in the right direction.

The panelists were asked their key philosophies for success. Geoffrey said know what your core is and then declare it, make it public. Find ways to optimize non-differentiating processes to free up resources for your core. Jim said to treat people as though you believe they can make a difference and supply challenging work to keep people engaged.

Next up in my SAS day was a panel on optimization. Bobby Hull of BGF Industries and Bill Nowicki of the Carolina Hurricanes were joined by Larry Mosiman of SAS on a panel hosted by Tammi Kay George. BGF is a leader in high-end composites and textiles. The Carolina Hurricanes, of course, are an NHL team. Optimization is, of course, the 8^th of the levels of analytics that Jim Davis was discussing. Optimization is used in lots of ways and Larry gave a quick overview. Optimization is a about finding the best of all possible solutions. In the context of marketing or collections it is about taking limited resources (offers that cannot be made to everyone) and finding the optimal pattern (where optimal means most profitable, resulting in most retention or whatever). Larry walked through how various approaches to assigning campaigns to customers produce sub-optimal approach. Just allocating first-come-first-served is inefficient. An explicit, rules-based decision is better but does not manage the trade-offs across large numbers of customers. Optimization focused on the whole problem and maximizes across the customer portfolio.

Bill, from the Hurricanes, discussed how they got started with optimization in ticket pricing. They had always done a lot of sales analysis to price tickets but needed to do more. Obviously tickets are a perishable item (with no value once the event passes) so maximizing revenue up to the moment of the game is important. Bobby, from BGF, was focused on optimizing the production of the various products which require different quality standards, have expensive raw materials and which are produced in high volume so mistakes escalate quickly. Besides problems with perishable items (like tickets or flights) and complex supply-side problems, Larry pointed out that optimization is increasingly used in areas like collections.

Tammi asked about the best outcomes from adopting optimization. Bobby said that the ability to relieve experts of the need to manipulate data allowed them to take information about the process and make it available to everyone. He also pointed out that this new project finally used all the data they had been warehousing – a classic example of getting more value from data once you focus on a decision. Bill agreed, emphasizing that a lot of what it did was show them the data behind the things they had previously suspected. The organizational change was really significant – starting to use data-driven decisions was a challenge. But the new approach allows the Hurricane’s to put their resources behind the games that will show the best return on that investment – optimizing the use of their resources.

Interesting that the optimization of ticket sales pushed discounts and deals earlier in the process. Obviously this avoids conditioning people to wait for last minute discounts. This is an interesting side effect of a more formal model of decision making. The optimized prices are used everywhere too – in promotions, at the ticket window and in the sales department. Both organizations reiterated that they had a lot of data that had been accumulated over the years but never really been used. Without a focus on the decisions they were trying to make the data just ended up being stored and accumulated.

Syndicated from BeyeNetwork

A little while back I got to spend a few minutes talking about analytics and optimization with Jack Mason of IBM. He posted the resulting video over on the Smarter Planet blog. Enjoy.

I am at Predictive Analytics World today so this is nice and timely. Look for some posts today on analytics.

I presented this morning on Advanced decisioning for process excellence and Sandy Kemsley wrote a nice summary over on her blog Column 2 : Advanced decisioning #GartnerBPM.

I gave the session again as a webinar and the recording is available here.

I am going to give the session as a webinar on Thursday of this week (details here) so if you missed it or want to recommend it to a friend, please do so.

Jim Sinur is up next on optimization and simulation. The world is changing fast so he sees the use of optimization and simulation becoming broader than its traditional role of improving existing process. Optimization and simulation allows:

• Try new processes in a safe environment
• Give business people power to try changes before they go live
• Help in scenario planning.

Key issues then are why optimization and simulation are important, how they help you survive and thrive and how you can get started.

Jim sees about 25% now using some kind of simulation v 5% a few years back and of them very few are using live data. Gartner defines optimization / simulation as using analytic tools and models to maximnize business process and decision effectiveness by examining alternatives before, during and after implementation and execution. Jim argues that we need to do more than we currently do – we need to be able to predict not just explain, do analytics inline not just offline and derive new events not just manage existing ones. But why is this important? We want to know how you can improve a process, add steps to it; or improve the decision making within the process; or make a more decision-centric process.

Companies can divide processes up into knowledge-worker, process-worker or computer-controlled processes and can use optimization and simulation to drive processes from expensive knowledge-worker processes to increasingly computer controlled ones. They can drive work to cheaper approaches and so free up people to work on higher-value work. You can monitor results and adjust policies as processes execute. You can, and should, develop and test scenarios to prepare and be proactive about changes.

Common uses of simulation are to try processes with test data and validate explicit assumptions. Companies are also using simulation to try different decision alternatives and process design changes. Optimization is coming too with predictive analytics, goal-directed processes, intelligent decision management with pattern recognition and data mining.

Simulation is the easiest place to start. Companies can create a sandbox to try things out to start creating a sense of the power of simulation. People can only use this if they are analytically minded and have access to good performance data. They need to be able to change process flows and rules, design and use dashboards and they need communication skills to share what they learn and persuade people to adopt it.

Gartner recommends building a simulation and optimization arhictecture incrementally. Logging process and rule changes as well as alerts of interesting events. Business Acitivity Monitoring, Data Mining/Analytics, rules and models are all required. Building a tool box to move from data/correlation/trends to understanding why with immediate feedback and an understanding of delayed effects. Ultimately it creates the ability to anticipate. But all this requires a culture change that focuses on actionable insight, sense and respond, alignment.

Pitfalls include technology that is not accessible to those who understand the business, lack of understanding of the concepts, invalid assumptions, solving for local optima and poor models.

Optimization and simulation contributre to survival because eliminate inefficiencies, find places that need attention and pre-test process and decision changes before deploying them. So, companies should focus on processes necessary for survival and use simulation to see how they could be improved. Soon, get focused on optimization in decision making and process design and make this part of the improvement cycle.

BTW Network problems are preventing me posting live from the show and meant I lost my whole keynote post – check out Sandy Kemsley’s post instead.

I got a quick overview of TOA Technologies recently. TOA Technologies was founded about 5 years ago to solve “the cable guy” problem – customers waiting at home for hours without knowing when the cable guy, or any other appointment, is going to arrive. Their core idea was that they would predict, with a high degree of accuracy, when the person will actually arrive while engaging the customer in the process, keeping them in the loop. They aimed to reduce customer pain by giving them more information, more regularly updated information and more accurate scheduling. They found that they also needed to understand in real time, what people were doing in the field and how long everything was taking. They needed therefore to collect this information and to develop a predictive engine that would use this information to predict how long activities will take to better schedule, route and allocate staff.

What they have is a SaaS platform designed for all the stakeholders in the scheduling problem – technicians in the field and other mobile employees, call center staff, customers etc. They accumulate information about what happens in the field and use that information to understand behavior and then apply that understanding. The product is action-centric and business rules driven. It is configurable for their clients in a menu-driven (no code) way. They capture the rules for a specific client and define decisions – what to do when. These can be based on data in the system or on the predictions being made. These decisions can include the preferences of end customers too – TOA clients’ customers can be empowered to enter their own rules – about notification or channel choice for instance. These decisions could be to reschedule an appointment, assign someone else, send notification etc. Client policies and preferences are stored as rules and some of these rules are configured by business owners at the client site, some by TOA, depending on the sophistication and level of complexity. End customer preferences are stored as data rather than rules (which is a pity as it limits what a customer can specify) and this data is combined with the client rules to drive decisions.

The system is designed to be self learning, picking up what has happened (or not happened) to continuously compare planned v actual. Users can always see what is happening and client companies can see what rules fired and why over time. The system learns across staff and event types and has a high granularity – allowing it, for instance, to learn that product X takes a long time to repair. A robust optimization tool is included for routing and scheduling. Rules are used as constraints and the optimization takes into account historical performance and up-to-date information as well as the rules configured in the system. All the technology has been developed by TOA – nothing is OEMed – and they take advantage of the SaaS model to enhance it quickly.

Each decision integrates customer and company rules, weighting the different pieces. I have talked before about this kind of personalization through include customers’ own rules but this is the first company I have spoken to that is really doing it. They like to say that there is a myth in field operations that allowing customers any control reduces efficiency. The folks at TOA maintain the opposite and argue, convincingly, that excluding customers from a decision in which they are clearly a major player degrades efficiency rather than enhancing it.

On Monday I posted about Enterprise Application 2.0 and promised to return with some thoughts on how to get from Enterprise Application 1.0 to Enterprise Application 2.0. Let’s see:

• Expose core elements as services
• Identify and manage processes – hook up legacy and new services into new, more effective workflows
• Find and automate decisions using business rules
• Manage simple events and correlate them into more complex, more meaningful ones
• Empower business users to manage rules
• Use RIA mashups to bring various interface elements together
• Establish ongoing decision analysis
• Apply analytics to optimize decisions
• Adopt experimentation and adaptive control

Much of this is pretty standard so let’s focus on the items missing from many legacy modernization checklists:

• Find and automate decisions using business rules
  The essence of decision management is the identification and management of operational decisions. These high-volume, transactional decisions are typically not handled well by Enterprise Application 1.0.  Identifying them explicitly, cataloging them and describing them using declarative business rules makes it possible to do much better. In a 1.0 world they could be:
  • Handled inconsistently (each systems does it differently, so the website offers a different price from the person in the call center for instance)
  • Handled manually (every transaction is put on someone’s worklist and left there until they come along and make a decision)
  • Handled only at a macro level (everyone is treated the same so no decision is really made at all, so all customers get the same cross-sell offer for instance)
  • Ignored (an opportunity to make a business decision is missed completely)
• Empower business users to manage rules
  Business users don’t want to manage rules any more than they want to write code. But if they can do those things as a side-effect of running their business then the systems will be more accurate, more current and more effective. Using technology like a Business Rules Management System allows for this kind of environment and brings IT and business people closer together.
• Establish ongoing process and decision analysis
  Most organizations automating processes realize that they need to do some ongoing analysis of how well the process runs and that process management is something ongoing not a once-and-done project. This is even more true of decision management. The best way to make a decision changes all the time because regulations do, because competitors do, because consumer expectations and preferences do and for many other reasons. Combining ongoing analysis of the effectiveness of decisions with business user empowerment makes it possible for Enterprise Application 2.0 to stay current and to ensure it keeps making good decisions.
• Apply analytics to optimize decisions
  Talking of good decisions, we need to make sure that we are putting our data to work in Enterprise Application 2.0. Using analytics (data mining and predictive analytics particularly) as part of our decision making is crucial. These techniques turn uncertainty about the future into usable probabilities and allow our applications to learn from all the data they collect. While these techniques can be used directly on processes and on event processing, they add the most value when applied to decision making. Analytics can turn all that historical data about what worked and what did not into something your decisions can really use so that future decisions are informed by the past.
• Adopt experimentation and adaptive control
  Last but by no means least we must have applications that help us manage uncertainty by letting us experiment and compare alternatives (adaptive control or champion/challenger). Even with the best analytics we cannot always tell what the best choice would be so we need to be able to experiment – to try several different choices and compare their effectiveness. It is not enough to do this in marketing or on a website (where it is often called A/B testing), it must be extended to core decisions. To find the best price, or the best offer collections strategy we must be able to compare the effectiveness of several different approaches.

Recapping the earlier post, let’s consider how we deliver the key elements of Enterprise Application 2.0:

• Agile and transparent
  Because the core logic of our business – the decision making – is managed and exposed in a business-friendly format using business rules our decisions are more transparent. Because business people can see how well the decisions are working and make changes directly when they are not our systems and processes are more agile.
• Empowers a flat organization
  Instead of referring and escalating decisions, putting them on a manager’s worklist, systems and processes make decisions so that front-line staff can take appropriate actions. Empowered staff must be given the options that are legal and shown what is likely to work and automating decisions enables this. With formal management and analysis of decisions, management can be sure how decisions are being made.
• Uses the data it accumulates to act analytically
  Organizations collect data in part to make better decisions. Applying decision management to operational decisions makes it possible to analytically enhance the decisions in day to day transactions, not just the executive suite.

And there you have it. Don’t forget I have a proposal in to speak at the Enterprise 2.0 conference and they have voting set up on their home page (http://www.e2conf.com/sanfrancisco/index.php). If you can grind through and find my proposal on Page 8 of 16 and then vote for it, I would appreciate it!

Syndicated from BeyeNetwork

I was attending IBM’s launch of its analytic appliances when it announced its intent to acquire SPSS. I did not get a chance to write much more at the time but I did not want to let the opportunity pass completely.I think the announcement represents a sea change in the decision management and analytics markets.

First it helps IBM deliver the predictive analytic element of its business analytics and optimization story. SPSS has many years of experience in productizing analytic R&D giving IBM a platform for bringing its own extensive analytic R&D effort to market. The potential for new capabilities based on IBM Research combined with new channels for PASW Modeler (Clementine as was) should be great for the analytics market as a whole.

SPSS has been a distant number 2 to SAS in the data mining/predictive analytics space for many years. While no-one,
including me, expects acquisition by IBM to drive them past SAS it does represent a huge opportunity. The new channels for SPSS’ products created by its acquisition by IBM include IBM’s worldwide sales force, obviously, as well as its extensive network of partners. The focus of IBM on strategic relationships with customers will, I think, be particularly valuable to SPSS which has a history of selling direct to analysts/modelers. While this direct-to-user approach results in lots of customers, it does not establish SPSS as “strategic” or establish a broad commitment to using the SPSS modeling tools. IBM is more likely to drive this kind of adoption. Not only is this good for the SPSS product lines, it is good for data mining and predictive analytics more generally as I think it will raise the profile of modeling in companies.

Beyond analytics, though, the more interesting aspect is the potential for IBM to put together a complete decision management platform. Having the IBM platform support decision management as well as process management, event
management and information management would be huge. And while acquiring all the pieces does not automatically give IBM such a platform they have historically worked to integrate their acquisitions fairly rapidly. I’ll blog more about this over on JTonEDM later in the week.

And when we take the two announcements (analytic appliances and SPSS together) we have some interesting business implications. The Smart Analytics Systems represent another potentially powerful channel for SPSS. The Statistics modules were already going to be on some of these as part of the Cognos installs but the deployment products
that SPSS has are ideally suited to this kind of appliance-based deployment. SPSS’ decision management products – PASW Deploy for risk decisions, marketing campaign decisions and inbound communication decisions – package up models and rules for advanced decision making. Thanks to the simple interfaces decision services like this have these
products are ideal for use in appliances. Putting these, and potentially other, decision-centric products on the Smart Analytics Systems will move them forward nicely.

Finally there is IBM’s services business. Some months back IBM announced a new service line in Global Business Service – Business Analytics and Optimization. While Cognos and ILOG’s optimization capabilities combined with various offerings from inside IBM R&D offered most of the software support this service line needs, the lack of an IBM branded data mining/predictive analytics offering was glaring. Adding SPSS now gives IBM’s BAO service line all the tools it needs. Not only will that help BAO, it will drive SPSS into more IBM accounts.

As I said at the time, the importance of IBM’s BAO service line should not be understated. Today high-end analytic solutions still require a significant amount of domain expertise and technical integration as well as multiple products. While I expect that to change, and the PASW Deploy products are an example of the kind of packaging that is required going forward, the ability of 4,000 IBM consultants to deliver more advanced analytics solutions is critical to
increasing adoption and awareness around the world.

For more on this consider James Governor’s post on IBM and SPSS, Forrester’s report on the acquisition, Neil Raden’s post on IBM’s vision for analytics or Merv Adrian’s post on IBM’s move into predictive analytics.

River Logic’s Enterprise Optimizer is what is increasingly known as an “Integrated Business Planning” solution. Enterprise Optimizer is designed to manage cross-functional decisions at strategic, tactical, and policy levels considering all the elements and consequences of those decisions. The models you build allow you to see the financial and operational impact of those decisions and then optimize them.

Enterprise Optimizer comes from work done by the University of Massachusetts with mathematicians from the Russian Academy of Science. The group had some background in AI, focused around trying to capture expert know-how to improve operational processes. From this research they moved to financial modeling, and over the last 15 years or so, have modeled over 200 different problems in various industries working with a range of partners.

The product that has evolved from this has recently been labeled by Gartner as an Integrated Business Planning (IBP) tool. IBP is defined as a collection of technologies, applications, and processes that connect planning functions across the enterprise to improve organizational alignment and financial performance. The technologies help companies understand, communicate, and manage constraints and consequences across the whole enterprise. The idea is not to just roll up numbers and pass them on but to have a more dynamic model of the connections. Unsurprisingly, they do a fair amount of work with companies adopting the Beyond Budgeting Round Table model.

The requirements for IBP include explicit process mapping (how a company creates value); financial modeling (ROI and forward-looking, activity-based cost, P&L ,and marginal opportunity analysis – all considering process constraints); a holistic view (products, customers, resources, supply chain processes, partners, etc.); and extensive optimization and business rules capabilities (objective function, rules, constraints, etc.). Plus collaboration, integration, and monitoring.

While Enterprise Optimizer is a horizontal technology, River Logic is focused on delivering EO-based solutions in a couple of areas, especially Consumer Packaged Goods with Healthcare as a secondary market.  For example, CPG solutions include strategy modeling (product portfolio, capital planning/network design), policy (inventory policy/product segmentation, sourcing, planning frequency), S&OP (executive, master planning, production planning, etc.), customer profitability, and cost to serve.

The product itself has a simple diagram style interface used to create the business processes that drive value in an organization. These diagrams model the supply chain and show how things like trade promotions impact volume, distribution and financial performance. Tactical planning solutions are constrained by policy, financial, and regulatory constraints from working capital to carbon emissions. The models also report forward-looking costs (akin to ABC costs but projected forward considering the constraints of the business), P&L, balance sheet roll-up, cash flow etc. Enterprise Optimizer models processes and more, but it doesn’t execute them – the model is just built and the engine figures out what the constraints and cost-drivers are.

The basic approach can be illustrated by considering a simple Purchase-Inventory-Conversion-Inventory-Sales process. The PICIS model is very common in manufacturing organizations – they buy raw materials (Purchase) that creates Inventory which is then manufactured (Conversion) into finished goods (Inventory) that must be sold (Sales). EO lets you easily create a process with a basic set of nodes, one for each step. EO will translate this model into a set of mathematical representations and run analyses against these nodes. Each node has a different representation and the user can specify different kinds of information for each node type. When the model is executed additional information is created on each node – the engine calculates things like opportunity value (e.g., the marginal profit from one more item or an additional customer) or optimal production schedules. Lots of information is defaulted, based on extensive research, so the model can be run quickly once basic information is filled in – users, of course, find it easier to edit a model once they can see what it does. As the user adds more information, the model becomes more constrained and more accurate and the tool is designed to support a highly iterative style of working.

The basic nodes support different elements of the business:

• Purchase nodes allow the price and constraints (min or max units available per period etc) to be specified for a user-defined list of raw materials. Once the model has been executed the node displays things like opportunity value (profit from getting one more unit of an item).
• Conversion nodes can specify different machines or resources that convert raw materials into finished goods. For each resource the user can specify their characteristics such as labor rate, fixed costs, period, and work units per period etc. Conversion nodes also can contain the processes that run on the resources. Process costs, rates, setup costs, etc. can all be specified. Various forms of cost analysis, activity-based costing, and throughput accounting are supported.
• Inventory nodes can specify the various products or materials and their price, etc. A flow from a conversion node to an inventory node allows you to map materials to the processes that produce them. The flow from raw materials inventory to resources allows you to specify the BOM or recipe for the various products.
• Sales nodes let you specify various constraints on sales, model price elasticity with non-linear constraints, etc.

There’s more, with each node supporting a potentially very large amount of information about the step, how it operates, and its financial implications. A fifth node type, financial report, can be added and mapped to a series of financial reports. The financial models can be specified in detail, but there is a lot of useful defaulting built in based on research with PriceWaterhouseCoopers.

Once a minimum amount of information is specified behind the nodes the engine can then be used to create a model of the business based on the specification. EO will optimize for profit on any unconstrained variable. Options to do detailed unit costs analysis and other kinds of analysis exist and can be added to the model run. Running the model updates the model, with implied attributes and optimized values, and these can then be updated as necessary by the user. EO also allows the models and constraints to be extended so that companies can model non-financial, non-process constraints, and measures like the number of truck trips through residential neighborhoods per day, special company measures, etc.

River Logic is also building an “IBP ecosystem” to make it easier for companies in the CPG and Healthcare spaces (initially) to deploy IBP solutions.

Excel, of course, is the major alternative and it is typically augmented by EO. Most EO users are doing what-if analysis and scenario planning – not real-time/workflow-oriented day to day optimization such as that done by folks using CPLEX, Dash, or Dynadec. The integrated financials and built-in; accounting best practices are, to my mind, the key differentiator, though EO also has the ability to compare scenarios using a web-based scenario management tool that allows users to name, store, retrieve, and compare entire models side by side.

Some time ago I saw this interesting little post -Recommendation Engine Secrets We Don’t Want You to Know: It’s not as Complicated as We’d Have You Think – that made the point that:

Most recommendation engines use one of a handful of methods that are well understood

And they are correct, of course. Recommendation engines involve some well understood elements:

• Data mining to determine significant customer segments, based on behavior
• Analytics to predict which products will be attractive to these segments
• Rules to enforce policies or regulations, determine pricing etc
• All packaged up into a Decision Service that takes in context information and returns a recommendation or set of recommendations.

The article went on to say that the issues with adopting recommendation engines were not in the “black box” that makes the recommendation but in cost, ease of setup/integration and customizability. Me I would add organizational change to that list (and put it first) as it is essential that you consider things like bonus or commission structures, marketing campaigns and more as you implement a recommendation engine.

One more thing – while people think of recommendation engines in a consumer-product environment there are, in fact, an almost infinite range of them. An engine can recommend policy coverages in insurance, delivery options in retail, suppliers in manufacturing, carriers in transportation and much more. All “recommendations”, all Decision Services, all useful.