I will be participating in the Analytics Executive Symposium being run on June 26th in Chicago by the International Institute for Analytics of which I am a faculty member.

The Analytics Executive Symposium is a unique and professional gathering of analytics champions who come together to hear and discuss the latest findings and research in analytics. The Symposium will feature the bi-annual meetings of IIA’s Analytics Research Councils (ARCs) for the Health Care and Retail industries. Special focuses on Analytics in the Enterprise will also be featured.
Please note: this is a by-invitation-only forum for IIA members.

Register here if you are an IIA member. To join contact the folks at IIA here.

Brenda Dietrich from IBM research wrapped up the morning with a discussion of some of IBM’s research. This involved both managing uncertain data at scale and driving analytics for this data. Projects cover systems of people, the future Watson, Outcome-based business and resilient business and services.

As everyone knows there’s a lot more data out there and it is both different from traditional data in format it is also fundamentally less certain. In fact IBM expects most data to be of this uncertain, unstructured data that has high Volume, high Velocity, Variety and issues with Veracity (because it arrives at different times or in different ways for instance). Uncertainty comes from process uncertainty (traffic patterns), “inherent” variability such as yield, data uncertainty (spelling and other editing errors, GPS uncertainty, ambiguity of labels, rumors and conflicting data) and filtering (we interpret or use data in a way that creates uncertainty).

The growth in this uncertain data does not mean you cannot make decisions. You can categorize the uncertainty, disambiguate, find methods of analysis that are robust in the face of ambiguity and more. You can use data to find and improve data, you can cross-reference data and consider it spatially to see how “close” it might be, you can consider time as the past influences the future but not the reverse. And, of course, the smaller your target segment the more data you need (forcing you to use this data).

Systems of people comes up next. Brenda asserted that traditional analytics systems were focused on process automation and  that the move to people-centric processes means you need a new approach to analytics. I am not sure about this as many processes now innovated by analytics were regarded as knowledge-centric, people-centric, “soft” processes. Nevertheless there are clear issues with using analytics to drive collaborative and people-centric processes. This requires three things:

• People enablement (adaptive, context-aware collaboration tools that keep track of what was done)
• People content (skills profiles, resumes etc in a structured way)
• People analytics (analysis of this new data to see what works, how teams might be formed etc)

Finally some thoughts on the future of analytics in three areas

• Explosion of unstructured data
• Skills gap, if only temporary, for both consumption and supply (not the same problem)
• Deploy analytics at scale

New data can come from an increase in decision-making scope or new formats/data. Broader scope is hard organizationally so much of the explosion comes from new data.

Analytic tools must expand to ingest and analyze new data sources and bring it into the analytic stack (feature extraction, entity identification etc). Plus you need to feed the results of decisions back into this analytic process for continuous improvement.

Analytic decision-making involves a series of steps:

• Data acquisition
• Filtering and extraction
• Core analytic algorithms
• Composition and packaging for
• Deployment

Research here involves bringing new and existing algorithms to a common platform while making the composition and packaging process hardware and deployment environment independent – removing the need to program it.

Finally some discussion of Watson. As IBM seeks new usages of Watson one of the key things is the ability to be more interactive, letting Watson’s algorithms ask for clarification or additional data while another is providing not just answers but also the reasoning and support for those decisions.

A whirlwind tour extracted from a longer presentation so this is a little choppy.

Steve Mills kicked off the IBM Smarter Analytics Leadership Summit. Business Analytics matter, he says, as shown by the focus of CEOs (8 out of 10 expect complexity to increase, enterprises applying analytics are more successful etc). The need for analytics is pervasive, with every industry seeing a massive expansion in the volume of data as well as an array of missed opportunities – such as the estimate $90B in missed sales because retailers don’t have the right products – and poorly managed risk – such as the hundreds of millions of dollars in healthcare fraud every day.

The key drivers of business analytics, he says, are the emergence of Big Data (price performance of hardware makes handling more data practical), the shift of power to the consumer (a focus on personalization and the increasing levels of social interaction between consumers) and continued pressure to do more with less (especially as the price of computing continues to fall). In response IBM is making massive investments – $16B for 30 acquisitions since 2005, 8 analytics solution centers, 10,000 technical professionals and 9,000 consultants and so on. Three example customers:

• Best Buy with their focus on registered customers and analyzing their behavior to drive an 8-10x improvement in advertising effectiveness to these customers while spending 5-7% less (opportunity creation)
• Alameda County Social Services using analytics to manage entitlements for citizens and identifying and eliminating fraud while better managing cases for a saving of $25M annually (fraud reduction)
• A major telco analyzing system logs to improve overall system reliability with real-time root cause analysis (risk management)

These three – opportunity maximization, fraud reduction and risk management – are the three classic use cases for analytics and for Decision Management.

Next up Mike Rhodin and Bridget van Kralingen to drill into some details of the new Smarter Analytics initiative. Mike kicked off talking about the need for analytics to be paired with Big Data and the general trend towards more intelligent, cognitive systems (the purpose of Decision Management, of course). Analytics and Big Data, he says, must be at the core of all the smarter planet solutions being developed across every industry. Big Data and analytics are enabling a new wave of front-office transformation. Solutions have moved from enterprise data to big data, from a business initiative to a business imperative and from something that transforms a single organization to something that transforms entire industries.

From a client perspective, IBM sees a new agenda among many C-suite executives who see analytics as both the biggest threat and opportunity on the horizon. In addition there is a move from scarcity of data and insight to an environment where there is an abundance of data and increasingly of insight. This means that driving to take action on this insight, Decision Management, will become critical. In addition professions in the front office will change as it becomes increasingly digitized – CMOs, for instance, must learn how to market analytically to succeed. This change is reflected in IBM’s survey data where the number of companies seeing analytics as a competitive advantage increased by 57% between 2010 and 2011 while those who identify themselves as using analytics to compete are more likely to be outperforming their peers.

Companies are moving from seeing analytics as a potential opportunity to something that is increasingly essential – something that is core to the company’s strategy and operations. CIOs in successful analytic adopters are taking the position that data and analytics are abundant resources and seeing how they can drive those into new solutions. In particular they are seeing big opportunities to grow retain and satisfy customers and increase operational efficiency.

To do this companies need to:

• Align their organization around data
  Create an organization-wide trusted platform for big data
• Anticipate
  Focus on predictive analytics at both the transactional and portfolio levels. Predict fraud, risk and opportunity
• Act
  Making real-time decisions where it matters – A Smarter Analytics Decision Management platform across text analytics, predictive analytics, optimization, business rules and entity analytics supported  by IBM’s Big Data platform
• Learn
  Then you need to learn so you can transform over time.

IBM feels it is in a unique position with its platform technologies, 20,000+ analytics projects and 9,000+ consultants, and a research program that includes Watson as well as many other analytic research projects. Their experience has also resulted in three focused Signature Solutions:

• Customer Next Best Action – a Decision Management solution
• CFO Performance Insight
• Anti Fraud Waste and Abuse – another Decision Management solution

Robert LeBlanc, head of the Middleware software group, came up to talk about the technology at the heart of these solutions. This technology has to deal with much large volumes of data from everywhere, data that changes more often – having higher velocity – and data that is of many new types an variety. This leads you to a platform that supports this kind of data environment – a Big Data platform – that handles both new and traditional forms of data and supports advanced analytics against it. This new environment will drive a need for new capabilities in much the same way that e-commerce and the web drove the creation of the WebSphere application server platform combined with a set of open standards and an ability to leverage previous generations of technology. Any Big Data platform must integrate with the current generation of data systems, must build on the open standards that exist and must be part of a robust ecosystem.

IBM sees this environment opening up new data sources (entity resolution from unstructured data for instance or network analysis) and new methods (adaptive analytics or optimization under uncertainty). New data sources and the ability to manage this data create new opportunities – 1.7B daily events at T-mobile or 1,000 buses being monitored in real-time across 150 routes in Dublin or sub-millisecond interventions to detect security intrusion in Brocade products. Of course one interesting example of this is in IT – the potential for using Big Data to improve the way IT operates across cloud provisioning, mobile enterprise and security.

IBM’s Big Data platform has several elements therefore:

• Information integration and governance
• Data Management:
• Hadoop for new data sources
• Stream computing (InfoSphere Streams) for data in motion
• Data Warehouse for traditional data at rest
• Accelerators layered on top of this data
• Capabilities for
• Visualization and discovery
• Application Development
• Systems Management
• Analytic Applications to make this easy to consume

They are also working to bring new partners into this space with over 100 new Big Data partners and are determined to create the best possible Big Data platform.

SAS sees Information Management as being about treating data as a strategic asset. Changes in competition, regulatory environment and more are driving this to be more importance. Information Management for SAS includes Data Management, Decision Management (mixing rules and analytics to drive operational decisions in decision services) and Analytics Management (supporting what I call an industrial analytic process). Governance and strategy must reach across all three.

Data Management first. Big Data and the increasing variety and velocity of data is making data management more complex. Companies need to be able to use transactional data, transactional event streams, text anlaytics and more to understand what customers are doing, for instance. SAS Data Management is supporting big data with forthcoming support for SAS / Access to Hadoop support from existing SAS languages and tools as well as a Proc Hadoop and support for Hadoop in SAS Metadata so that you can see what data is in Hadoop. SAS has also added some support in the ETL suite for Hadoop, allowing some transformations to/from Hadoop.

Four themes in business visualization:

• Consumer-oriented BI
  Walk up and use, understanding through visualization, self-service
• Deriving value from big data
  Analytics and visualization that scales with in-memory
• Approachable analytics
• Integration into the user’s world
  Outlook, Office, mobile

The new product (coming in March) has four components:

• Environment Manager for setup and monitoring
• Visual Analytics Explorer for ad-hoc analysis and discovery
• Visual Design for report design (web or mobile)
• Mobile BI with native interactive reports on iOS and Android, both generic and task-specific

Dr Goodnight kicked it off the executive viewpoint at the 2012 SAS Inside Intelligence session. SAS had another growth year in 2011 ($2.7B and growth of 12.1%) as Jim highlighted both its continued investment in new buildings around the world and the fact that many SAS locations around the world, and SAS overall, continue to get strong best place to work ratings (this is important to Jim personally as well as for recruitment).

Jim focused on Big Analytics and Big Data as keys to SAS future success. SAS has been working with customers over the last few years to expand existing SAS technology to support multi-processor and multi-core environments as well as large in-memory computing environments. This investment in high performance analytics is ongoing and is helping customers take jobs that took more than a day (30 hour markdown optimization) or could not complete before the new work day (18 hour risk analysis) and drive them down to hours or minutes. SAS is now committed to moving all its algorithms to this high performance analytics environment.

A big part of this effort is SAS’ work with vendors like Teradata and Greenplum to get in-database scoring and in-database modeling. Another key element is the simplicity of this – simply add “hp” to a procedure to push it from the default server and push it to the in-database or other HPC environment you have available. They are also applying it to traditional BI capabilities using Visual Analytics Explorer, a new product shipping in March, that gives you visualization and query tools against an in-memory model that might contain billions of rows.

After some great demos of the new visualization products, Jim Davis came on stage to finish the executive overview. Jim reiterated the renewal model for SAS’ revenue and the importance of demonstrating continued value to existing customers. This model is important to them and they see continue strong renewal rates in 2011. New customers matter too, of course, and many regions showing 20%+ growth in new customers. Meanwhile SAS continues to invest nearly 25% back into R&D – and that translates into a lot of R&D dollars. Revenue is balanced with the US only being 40% while EMEA is 42%, for instance, with Asia Pacific coming up at 12% (renewal model makes changes take time of course). The industry picture not so balanced, with 40% Financial Services as you would expect.

SAS showed growth across the product portfolio with customer intelligence and fraud detection growing fastest backed by supply chain and retail. Of course supply chain and retail were starting from low bases but the fraud and customer intelligence numbers are impressive. SAS has also been investing in some analytic centers of excellence around the world (Cary now with Europe, Asia coming) with hosting as well as analytic expertise growing to over $100M a year now. Cloud-based solutions are clearly hot as we found in our Predictive Analytics in the Cloud study. Meanwhile SAS is expanding its channel partners as most of its revenue continue to comes direct.

Jim highlighted some of the SAS sub brands like DataFlux/baseline consulting, jmp, RiskAdvisory, vsti, Memex, IDEAS etc. Some of these businesses grew strongly last year (jmp, Memex), others (Assetlink) are rolling into the main SAS brand and others are sharing sales teams with the main SAS business (DataFlux). They just added aiMatch for ad intelligence.

Jim highlighted five themes for the rest of the event (my emphasis):

• High Performance Analytics
• Business Visualization
• Information Management
• Decision Management
• Cloud

High Performance Analytics is a key focus area for SAS. SAS Grid Computing, SAS In-Database, SAS In-Memory Analytics make up the high performance analytics environment. This is clearly a big investment area for SAS with lots of investments. Their approach is to develop a platform that can be used across their tools, analytics (from statistics to predictive analytics and optimization) and analytic applications. The in-memory analytics product, for instance, works with Hadoop, Greenplum, Teradata or a non data server. They have launched a high performance microsite for this.

Mobile is another focus area, with lots of “fit for task” interfaces specific to applications and tasks.

KNIME is an open source data analytics product based in Zurich, Switzerland that I last wrote about a couple of years ago. They have been working away on the product since then (having started development in 2004 and released their enterprise components in 2010) and have been refining their business plan at the same time. They now have 9,000 registered users/organizations and over 500,000 downloads. About 50% of these are in life sciences and 50% in BI and analytics – this represents a significant increase in non-life sciences usage. As an open source product they have 50 or more very active community developers and the company itself is 15 people (about half are at the University of Konstanz) and describes itself as “small but profitable”. Since I last spoke with them they have added a number of enterprise components and continued to develop the core open source platform.

The platform is based on Eclipse and has, in particular, been extended with lots of new node types. Like most modern analytic workbenches KNIME allows a modeler to define the sequence of steps to produce a model. The various actions that can be performed as part of the modeling process is determined by the available node types – nodes to retrieve data, process and transform that data, apply modeling algorithms (both established and more esoteric) or output results.

In addition a new decision tree viewer has been developed and data support (various databases, SAS and SPSS, and other flat files) has been improved. KNIME supports PMML generation for models including the most frequently used preprocessing operations required for a model – this was added to PMML in 4.0 and KNIME is one of if not the only vendor completely supporting this. There are lots of nodes for pre-processing and many algorithms including some of their own, some from the R Project, others from Revolution Analytics and Weka as well as lots of new visualizations.

One of KNIME’s strengths is a well defined node interface that is nicely self-contained allowing easy integration of new node types without destabilizing the whole environment. KNIME makes it easy to add nodes so that community members and partners can extend the tool with their own node types. A community site manages the community submissions and makes it easy to add any you want. Data management and execution control are also handled through well defined interfaces allowing people like Pervasive DataRush to add their own integrations.

Because KNIME is built in Eclipse they also integrate with other Eclipse projects like BIRT (an open source reporting framework). This allows these open source reporting components to be added to KNIME workflows.

Improvements and new node types are often sponsored by companies that use the product, though KNIME continues to push its own development plan. KNIME is also an integration platform for many vendors with 15 technology vendors integrating with KNIME, especially in the life sciences environment where life science companies are paying their technology vendors to integrate with KNIME to create a single environment.

The KNIME workbench is open source with registration desired but not required. This open source client version is a complete environment. Access, transformation statistics and data mining, visualization, reporting and workflow is all available in the open source with 1,000+ native and embedded nodes including text processing, social network analytics, image processing and more. Companies can add support to the open source project or adopt the server or enterprise components that are proprietary and licensed.

KNIME Team Space allows a small group to have a shared repository. KNIME Server adds the full server-based capabilities including remote and scheduled execution of modeling flows, a workflow repository, shared data in a managed repository, shared metanodes or component workflows for reuse, and web browser access. The browser interface exposes workflows to users who can enter the parameters defined for the workflow and get a result displayed back in the browser. An API also allows “headless” execution of workflows. With the full suite companies can write new nodes and wrap legacy software, then allow power users to develop templates and shared workflows, provide this to scientists or others who adapt and use these workflows as well as managers who just run pre-configured ones.

The company is shifting from selling directly to life science and a small number of other companies to an approach based on partner channels with domain expertise. Target markets are obviously life science as well as data mining/predictive analytics (customer intelligence especially). KNIME’s value proposition is in its openness as well as the ability to reduce the cost for existing heavy users of analytic workbenches while also targeting smaller companies that can’t afford the large vendors.

Besides the partnerships with Pervasive and BIRT noted, KNIME has a strong partnership with Zementis for PMML execution in database and on servers and with Dymatrix, a Europe-based partner offering model monitoring and automated updates.

KNIME will be one of the vendors listed in the forthcoming report on Decision Management Systems platform technologies.

I am giving the closing keynote on day 1 of Predictive Analytics and Business Insights 2012 in San Francisco on February 9th at 3:30pm. I will be speaking on “The Intelligent Enterprise: Utilize decision management to move from Business Intelligence to Predictive Analytics” and attending the networking reception afterwards. Other speakers at the event include Saum Mathur – Vice President, Information Technology, Global Business Intelligence at Hewlett-Packard, George Roumeliotis - Leadership for Advanced Analytics and New Product Development at Intuit, and Nurtekin Savas – Director of Strategy & Analytics at Visa.

You can get details on the event here.

In a recent article over on All Analytics – Analytics-Business Alignment Needs Work – Beth Schultz discussed a set of Gartner predictions written up by Doug Laney. In particular she highlights Gartner’s finding that companies

cited …aligning BI initiatives with corporate strategy and objectives nearly three times as often as they called out technology-related issues

This is a problem I see regularly across both BI and other analytics efforts. I think the primary driver of this is an abject under-definition of the decisions that BI and analytics are supposed to be enabling/improving. We model and manage business processes and the data they need. We don’t model or manage the decisions within those processes nor consider their impact on the business. I call this process Decision Discovery and wrote about it extensively in Decision Management Systems: A Practical Guide to using Business Rules and Predictive Analytics - the whole of chapter 5 is devoted to this critical first step. I wrote a quick overview recently about the use of  decision discovery to gather requirements for advanced analytics and my experience is that it also works for driving successful BI efforts. Organizations that understand the decisions involved, how they relate to each other, what information and know-how is needed for each decision and how decision decompose into smaller, more focused ones are more likely to see where analytics will make a difference. By evaluating the impact each decision has on business objectives and key performance indicators, the alignment of analytic efforts to corporate objectives can be achieved – they are linked by the modeled decisions.

In addition, Doug is quoted as saying

Too often we observe organizations developing and deploying hindsight-oriented reports and/or query applications focusing on metrics that users may find interesting, but that don’t represent the operational or strategic controls used to facilitate business performance.

Once again the linkage of decisions to objectives helps as this allows the analytics to be prioritized and focused on assisting with those decisions that really matter to the key objectives of the organization.

In his recommendations Doug has one that particularly resonated with me:

…a top-down view of metrics, which should determine and define your tactics

And this top-down view of metrics should be combined with a business-centric, top down view of decisions.

I also liked

Make experimentation and closed-loop implementations standard practice

As the third step in my approach is always what I call Decision Analysis – the ongoing process of improvement, test and  learn and decision performance monitoring.

Decision Management, and the discovery and modeling of decisions in particular, really drives business/analytics alignment.

If you are interested in how to go about decision discovery or what to build you own decision inventory, drop us a line info@decisionmanagementsolutions.com and we will see if we can help. You might also want to check out my forthcoming workshop on Business Friendly Data Mining at Predictive Analytics World.

Quiterian is a Spanish company with offices in the US, Mexico and Europe. Quiterian Analytics aims to be complementary to traditional tools for reporting by helping companies get more value from their data sooner. In particular they aim to help companies anticipate the future by providing simple to use predictive analytics and by empowering users while reducing IT costs. They see the BI market in three broad segments:

• Dashboards and reports – traditional BI tools as well as some of the newer, easier to use visualization tools – with little or no focus on advanced analytics
• Data mining tools focused on professional data miners and statisticians, typically not terribly visual.
• Visual Data Mining tools that site between the two aimed at analysts and business users with no dependency on IT or data miners. This is where Quiterian is targeted

They offer fast and comprehensive data analysis and visualization, self-service for non-technical users and aim to be very agile with no need to pre-create cubes etc. Specifically Quiterian Analytics is a web-based SaaS tool that includes functions for

• Its own analytical database that is a columnar/in memory hybrid
• Data acquisition, exploration and engineering
• Advanced analytics
• Distributing and sharing findings
• Workflow

Users can identify various data sources and load them up using Quiterian’s ETL. Drag and drop exploration tools allow users to profile the data, examine the underlying records etc. Charts of distributions, various statistical measures and frequency distribution can be displayed easily. Subsets can be selected from the graphs and everything recalculated based on those subsets.

Pivot tables, venn diagrams, bubble charts and more can be used to see what is going on in the data. For instance the customers who don’t have a credit card but do have a current account and don’t have a credit card from another company. These analyses can be saved, exported and shared in a variety of ways.

Expressions, calculated values, can be defined and used in analysis. Users can also use more advanced tools such as aggregates across rows, tools to manage quartiles/deciles, or tools to apply Pareto distributions to see what percentage of profit comes from a certain percentage of customers for instance. Profiling tools can be used to assess correlation, for instance by identifying which attributes are strongly correlated with profitability. A profile can then be applied to a group based on a previous query or analysis. New selections can be based on these profile attributes and pivot tables and other kinds of analysis can build on these selections.

In addition the tool supports decision trees, clustering and forecasting for more advanced analytics. The decision tree data mining algorithm has some nice usability features with color coding, accuracy measures etc. This can then be saved as an algorithm and be applied to any set of records selected in the tool using any of the other elements.

Finally the workflow component allows tasks within the tool to be scheduled. You can schedule actions to take when a new model is created, as part of a scheduled campaign, when specific technical changes are made or just at a particular time. These tasks can apply models or algorithms, send emails etc. Quiterian is working on adding some specific action types to expand this e.g. invoking actions in salesforce.com or email marketing tools as well as twitter, SAP, Omniture and others. As the range of actions expands, the ability to embed these analyses into decisions will likewise expand. A PMML import/export capability is also in the works.