≡ Menu

DataRobot is focused on automated machine learning and on helping customers build an AI driven business, especially by focusing on decisions that can be automated using machine learning and other AI technologies. DataRobot was founded in 2012 and currently has nearly 300 staff including 150+ data scientists. Since it was founded, well over 200M models have been built on the DataRobot cloud.

DataRobot’s core value proposition is that they can speed the time to build and deploy custom machine learning models, deliver great accuracy “out of the box” and provide a simple UI for business analysts so they can leverage machine learning without being a data scientist. The technology can be used to make data scientists more productive as well as to increase the range of people who can solve data science problems.

DataRobot runs either on AWS or on a customer’s hardware. Modeling-ready datasets can be loaded from ODBC databases, Hadoop, URLs or local files – partnerships with companies like Alteryx support data preparation, blending etc. The software then automatically performs the kind of data transformations needed to make machine learning work – data cleansing, feature engineering needed for the various machine learning algorithms such as scaling and converting data to match the algorithms. It does not currently generate domain-specific potential features/characteristics from raw data, instead making it easy for data and business analysts to create them and feed them into the modeling environment. Once data is loaded, some basic descriptive statistics are loaded and the tool recommends a measurement approach (to select between algorithms) based on the kind of data/target.

DataRobot can apply a wide variety of machine learning algorithms to these datasets, for now almost exclusively supervised learning techniques where a specific target is selected by the user. Multiple algorithms are run and DataRobot partitions data automatically to keep holdout data for validation (to prevent overfitting), applies smart downsampling to improve the accuracy of algorithms and allows some other advanced parameters to be configured for specific kinds of data. Once started, DataRobot looks at target variable, dataset, characteristics, combinations of characteristics and selects a set of machine learning algorithms/configurations (blueprints) to run. These then get trained and more “workers” can be configured to speed the time to complete, essentially spinning up more capacity for a specific job.

As the algorithms complete, the results are displayed on a leader board based on the measurement approach selected. DataRobot speeds this process by running the blueprints initially only against a subset of the data and then running the top ones against the full dataset. Users who are data scientists can investigate the blueprints, see exactly the approach taken for the blueprint in terms of algorithm configuration, data transformations etc. Key drivers- the features that make the most difference – are identified and a set of reason codes generated for each entry in the dataset. Several other descriptive elements, such as word clouds for text analytics, are also generated to allow models to be investigated.

The tool also has a UI for non-technical users. This skips the display of the leader board and internal status information and displays just a summary of the best model with its confusion matrix, lift and key drivers. A word cloud for text fields and a point and click deployment of a scoring UI (for batch scoring of a data file or scoring a single hand-entered record) complete the process. More advanced users can interact with the same projects, allowing the full range of deployment and reuse of projects created this way.

Once a model is done, the best way to deploy them is to use the DataRobot API. A REST API end point is generated for each model and can be used to score a record. All the fields used in the sample are used to create the REST API and the results come back with the reason codes generated. Everything to do with modeling is also available through an API, allowing customers to build applications that re-build and monitor models. Users can also generate code for models but this is discouraged.

You can get more information on DataRobot at http://datarobot.com

The Rexer Data Science survey is one of the best and longest running polls of data mining, analytic and data science professionals. I regularly refer to it and blog about it. It’s time to take this year’s survey – and the survey is aimed at all analytic people, no matter whether they consider themselves to be Data Analysts, Predictive Modelers, Data Scientists, Data Miners, Statisticians, Machine Learning Specialists or any other type of analytic person. Highlights of the 2017 survey results will be unveiled at Predictive Analytics World – NY in October, 2017 and the full 2017 Survey summary report will be available for free download from the Rexer Analytics website near the end of 2017.

The survey should take approximately 20 minutes to complete. Your responses are completely confidential.

Direct Link to Start the Survey – Access Code:  M4JY4
Karl tells me it is OK to share this Access Code with your friends and colleagues as it can be used by multiple people. You can also get more survey information & FREE downloads of the 2007-2015 Survey Summary Reports from Rexer Analytics.

SAP BusinessObjects Predictive Analytics 3.1 is the current release of the SAP predictive analytic suite. Like most in the analytics space, SAP sees its clients struggling to make use of massive amounts of data that are newly available while facing ever increasing business expectations, faster business decision cycles and an analytical skill gap. SAP therefore is focused on predictive analytic capabilities that:

  • Produce accurate results in days not weeks
  • Deliver operationalization for machine learning at scale
  • Embed predictive analytics in business processes and applications

The predictive analytics suite consists then of four elements:

  • Data Manager for integrating and creating and reusing (potentially very wide) datasets
  • Automated Modeler, a wizard-driven modeling tool for predictive analytics
  • Predictive Composer, a more custom pipeline/workflow development tool for maximum control
  • Predictive Factory to operationalize all of this

These can access data from SAP HANA, SAP VORA, Hadoop/Spark, 3rd party databases and SAP HANA Cloud. And they can be embedded into SAP applications and other custom applications.

Four offerings package this up:

  • SAP BusinessObjects Predictive Analytics Suite for on-premise and for on cloud
  • SCP Predictive Services on cloud for embedding machine learning
  • SAP BusinessObjects Predictive Analytics for OEM/SAP application embedding

SAP is focused on speed, building models fast, but also on automating techniques. The assumption is that organizations need to manage hundreds or thousands of models and very wide data sets. Plus, for many SAP customers, SAP integration is obviously important. Finally, the suite is designed to support the whole analytic lifecycle.

The tools are moving to a new UI environment, replacing desktop tools with a browser-based environment. Predictive Factory was the first of these and more and more of the capabilities of the suite are being integrated, allowing Predictive Factory to be a single point of entry into the suite. As part of this integration and simplification, everything is being built to be effective with both SAP Hana and Hadoop. There is also an increasing focus on massive automation e.g. segmented modeling.

One of the most interesting features of the SAP BusinessObjects Predictive Analytics Suite is that there are two integrated perspectives – Automated Modeler and Predictive Composer. This allows data scientists and analytics professionals to build very custom models while also allowing less technical teams, or those with more projects to complete, to use the automation. All the models are stored and managed in Predictive Factory and Predictive Composer can be used to configure nodes for use in Automated Modeler. Predictive Factory also lets you create multiple projects across multiple servers etc. Existing models can be imported from previous tool versions or from PMML, new tasks (such as checking for data deviation or retraining models) can be created and scheduled to run asynchronously. Tasks can be monitored and managed, allowing large numbers of models to be created, supervised and updated.

The same automated algorithms can be accessed from the SAP BusinessObjects Cloud. Users can identify a dataset, identify something they are interested in and run automated modeling algorithms to see, for instance, what influences the data element of interest. This requires some understanding of the power and limitations of predictive analytics but no skill with the analytic technique itself. Data is presented along with some explanation and supporting text. The results can easily be integrated into stories being developed in the BI environment or applied to datasets. Over time, this capability will include all the capabilities of the on-premise solution.

Predictive Analytics Integrator allows these capabilities to be brought into SAP applications such as SAP Fraud Manager. Because SAP applications all site on SAP HANA, the Predictive Analytics Integrator is designed to make it easy to bring advanced analytics into the applications. Each application can develop a UI and use terminology that works for the application users while accessing all the underlying automation from the suite.

Predictive Analytics 3.2 in July will be the first release where the suite’s components are being integrated into the browser environment and the Predictive Composer name will be used. This release will not have 100% equivalence with the desktop install but will support the building and deployment of models using both the data scientist and automated tools.

You can get more information on the SAP BusinessObjects Predictive Analytics Suite here.

I recently worked with Tho Nguyen of Teradata on a white paper called Illuminate Dark Data for Deeper Insights

While organizations of all sizes across all industries are keen on becoming data-driven, most focus on only a fraction of the many types of available data. Not accessing a fuller spectrum of data, including those from “dark data”—emails, texts, images, photos, videos, and other documents—along with traditional data sources can limit an organization’s ability to gain a complete picture of their customers and operations, and exclude them from game-changing insights that improve business outcomes.

Dark data is defined by Gartner as “…the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes…”  (Gartner IT Glossary) and this paper discussed what makes data dark, what kinds of data go dark and how new technologies are illuminating this dark data.

You can register for the paper here.

If you want to talk about the role of dark data in your analytic and decision management systems, drop me a line.

Reltio Cloud is a modern data management Platform as a Service (PaaS) company focused on delivering data-driven applications, founded in 2011 by folks from Siperian which was acquired by Informatica. Unlike most data integration and MDM platforms, which are IT-focused, Reltio’s mission to make it possible for business and IT teams in enterprises to “Be Right Faster” by building data-driven enterprise apps that deliver reliable data, relevant insights and recommended actions. They compare these applications, based on broadly sourced, cross-functional data, with the traditional approach that delivers process-driven and siloed data. With data-driven applications contextual, analytical and operational data can all be brought together. This requires a reliable data foundation.

Reltio Cloud is a modern data management Platform as a Service (PaaS) and it includes:

  • Master Data Management as the core for delivering a foundation of reliable data
  • Predictive Analytics and Machine Learning through the inclusion of Apache Spark in the platform
  • A Graph Model allows for network and analysis integration across highly variable data sources
  • Big Data Scale and Performance so that transaction and newer data can be managed not just customer data
  • Workflow and collaboration capabilities to manage and curate data
  • Data as a Service is core to the platform so that third party data services can be easily integrated.

The graph schema is key to Reltio, allowing them to store both entities and their relationships in a semantically rich way. Data is stored in a combination of Apache Cassandra, graph technology, and in-memory structures such as Elastic. It offers an extensible structure for an organizations entities and relationships. The Reltio cloud collects data from multiple sources, matches, merges and relates them to create these relationship graphs and these graphics then underpin the data-driven applications being developed.

Reltio Insights shares objects (built from the profile and transaction data) with Reltio Cloud and analytics environments like Spark (either the Reltio platform or a customer’s own) to create analytic insights. These insights then get integrated with the master data so that these can be made available to data-driven applications. Reltio Insights is designed to rapidly provision master and transactional data into a Spark, environment. The resulting analytic insights are available throughout the Reltio environment, added to the data e.g. a customer’s churn propensity becomes an attribute of the customer profile.

The applications themselves can offer several different views – for instance, some users such as data stewards might see where the data came from and be able to interact with it to clean it up while others might only see the final, integrated view. A standard feature of the app is to visualize relationships, based on the underlying graph models. Some simple analysis, such as distribution of transactions by channel, can be easily included as can the results of more sophisticated analytics. Anything available in the Reltio data platform can be collaborated upon, managed and updated through data-driven operational applications. The data can then be used to drive analytical model development and provision the data to other operational applications. In addition, everything is tracked for audit and change purposes and the workflow engine can be used to manage requests for updates, changes etc.

Everything in the platform is available as HTML 5 widgets so that additional content like Google maps, can be easily embedded, and this means that Reltio content can also be easily embedded elsewhere. Many customers take advantage of this to mix and match Reltio content in other environments and vice versa. Similarly, all the data in Reltio Cloud is available from a REST API for use in all legacy operational and analytics systems.

You can get more information on Reltio here.

DecisionCAMP 2017 is coming up July 11-14, 2017 at Birkbeck College, University of London. This is going to be a great opportunity to learn about decision modeling, the Decision Model and Notation (DMN) standard and related topics. In fact the week is full of great things to do if you are in London or can make it there:

You can register for DecisionCAMP here.

One of our clients was presenting recently at a TDWI conference and was picked up on TechTarget – Analytics teams give data science applications real scientific rigor. It’s a great article with some good tips about using a repeatable methodology like CRISP-DM, especially when combined with decision modeling as a way to capture business understanding and drive collaboration (see this post too on Helping your analytic projects succeed with decision modeling and CRISP-DM). As Anu Miller of Cisco put it

We ask each other all the time, ‘What business decision are you looking to support?’

This focus on method and on business decisions also helps bring teamwork across the business/data science team divide too. As she went on to say

Those things almost force you to be collaborative. There are no unicorns on our team. We have to work together.

All good advice. If you live in the Bay Area, you can hear me talk about some of the key aspects of this approach at the Global Big Data Conference when I talk about ‘Don’t Apply Big Data Analytics To The Wrong Problem: Put Decisions First’. If you don’t live locally, check out this case study: Bringing Clarity to Data Science Projects with Decision Modeling.

And remember, if you would like to talk about improving your data science approach or other help operationalizing your analytics, get in touch.

Equifax has been expanding beyond credit bureau data in recent years by providing better access to a broad range of their own fraud, employment, wealth, commercial and alternative data sources as well as 3rd party data to position themselves as an insights company. As part of this focus, their Decision Management platform, InterConnect, was rebuilt from scratch as a foundation for cloud-centric and multinational decisioning applications. InterConnect is designed to support a broad range of decision management solutions, with an initial focus on customer acquisition.

InterConnect is designed to be a secure cloud-based decision management platform to define and execute decision policies at the front line. It is focused on delivering robust data, powerful decisioning and streamlined technology.

There are four main decisioning tools in the platform

  • Insight Gateway
    Streamlined transactional access to diverse data sources.
  • Attribute Navigator
    To manage and deploy the data catalog and derived attributes.
  • Model Integration tool
    A single tool to integrate, audit and deploy predictive analytic models into production.
  • Rules Editor
    A rules management environment for creating, testing and optimizing business rules

These four decisioning tools are presented in a common decision portal that is role-based, so only selected elements are exposed to users. This portal is gradually becoming the presentation layer for all Equifax services.

Insight Gateway gives real-time transactional access to data sources. This includes many standard data sources (such as Equifax’s own credit bureau data) as well as specific local data sources developed in each country  Insight Gateway uses a microservices architecture, JSON and self-description to make integration flexible and visual. It is supported by a Data Provisioning Tool that allows for discovery and conditional orchestration/integration.

Attribute Navigator allows users to create a catalog, define the attributes, test and deploy. It supports the definition of custom attributes against multi-bureau data, third party data or customer data. Customer data may be hosted by Equifax or can be sent on each individual transaction. The environment supports testing and auditing of attributes.

Model Integration Tool lets teams integrate, audit and deploy predictive analytic models. It supports scorecards as well as decision trees and ensembles. It can guide users through the integration of SAS, SPSS and R models to generate PMML ready for execution in the platform as well as a specification document for compliance and governance. This generated PMML, or other PMML models, can be executed in the Interconnect platform using the well-known Zementis engines (both ADAPA for cloud deployment and the Zementis Hadoop plugin – reviewed here).

Rules Editor is based on the modern rules platform provided by Sparkling Logic – SMARTS (most recently reviewed here). This provides a data-driven rule editor with authoring, visualization, testing and simulation all in one interface. The rule authoring environment supports cascading and inheritance, rule flow management, champion/challenger, trace execution and reports on key performance metrics.

Configuration of the four services for individual customer requirements and solution orchestration at runtime is delivered by Equifax’s professional services. InterConnect can be custom-designed or accessed as a pure platform utilizing all or individual service as needed. It is available in the US, Canada, UK, Spain, Peru, and Chile. Argentina, Paraguay, Uruguay, Mexico, Australia and India are expected to be added in the future.

You can get more details on the platform here.

Humana presented at InterConnect 2017 on their use of business rules on z/OS. Humana is a 3M member health insurer and a user of IBM Operational Decision Manager (ODM), IBM’s Business Rules Management System and has been focusing on using it to modernize some of their key mainframe systems – something that Humana is focusing on as part of its efforts to reuse existing assets. ODM runs on IBM z/OS for batch, CICS, standalone rules or WAS on z/OS, allowing them to run business rules on their mainframe systems. Using ODM allows Humana to reuse these assets while also transforming their development approach to be more responsive, more aligned with the business and more consistent to ensure compliance and manage risk.

Humana uses ODM for Medicare, Claims, Enrollment and Dynamic forms:

  • Humana has 700 Medicare plans that have to be reviewed for CMS compliance. A .Net application integrated with the decision service answers the CMS questions with an SLA of 2 seconds. The environment allows the business to manage the 1,700 rules in the application using ODM Decision Center. This improves the change cycle from months to weeks.
  • Claims processing handles multiple procedure payments and member cost sharing, for instance. Run as Cobol CICS and batch systems with 500+ rules and decision tables. 3.5M ruleset executions daily. Manual rules that could not be coded in COBOL now in ODM, increasing the rate of STP and driving savings. Savings in first week exceeded cost of development!
  • Enrollment for large and small group – about 30+ rule applications to reduce enrollment from a week to real time.
  • Dynamic forms is for authorization, generating custom questionnaires dynamically. 70+ questionnaires can now be developed and tested by the business. Complete audit trail and ability to make rapid changes have been key.


  • Humana runs ODM on z/OS.
  • Rule Designer (IDE) is used develop the vocabulary, templates, rules etc. This is tested and then pushed to Decision Center for ongoing business user management.
  • Decision Center is used across all environments. This allows business user engagement in a thin client environment and can be synchronized with the technical environment. Decision Center supports deployment, versioning etc and becomes the single source for rules. The support for testing and simulation are key and very popular with business users. They use both the Enterprise and Business Console, though the Business Console is the standard go-forward environment. All this runs on Z Linux, letting them take advantage of the integration of Z and DB2, the power of Z etc.
  • Decision Center is used to deploy Decision Services to the various environments Humana used – z/OS,Linux on Z etc.
  • The Rule Execution Server Console is used to monitor executing rules, trace rule execution and manage the deployed services.
  • They take advantage of the Java/DB2/z integration and performance tuning to maximized the performance of their rule execution. They mix and match decision services deployed for interactive or batch services, integration with COBOL or CICS etc etc. Lots of options for integrating the decision services into the mainframe environment.

Moving forward they are looking at some changes for high availability workload management as well as embedded batch for improved performance. Plus they want to complete the move to proper Decision Services from traditional rule applications.

Overall ODM on z/OS has delivered several benefits:

  • Cost savings
  • Improved time to market and business engagement
  • Single source of rules
  • Incremental adoption

State Farm presented at IBM InterConnect 2017 on their challenges with large scale deployment of IBM Operational Decision Manager (ODM) – IBM’s Business Rules Management System. State Farm has a set of specific things it wants out of its BRMS:

  • Well defined artifact lifecycle for auditing
  • Rigorous deployment process support for confidentiality and consistency
  • Self-service so authorized users can work 24×7

This is a pretty technical topic as State Farm is a large scale user of rules and IBM ODM. They have >500 rule projects with rulesets that vary from 100 rules to 10,000, some invoked every few minutes to very large volume batch jobs. Some of the decisions are trivial but others have big legal implications and must be 100% right. 45 different teams with 430 users of Decision Center are working on projects with over 80 deployment targets on Linux and Z/OS hosts.

They need RuleApps – the deployable units – to have well defined content, be accessible, controlled on need-to-know and governed appropriately for its criticality. Each RuleApp version is built once to ensure consistency and decouple deployment from the Decision Center editing environment. They are then promoted through the test and production servers. Its also important to manage the system users and business users appropriately.

Key precepts then:

  • RuleApp builds that are promoted for real use come from Decision Center
  • Well-formed RuleApp version baselines to track content
  • Self-service tooling to manage RuleApp, XOM, builds etc
  • Keep users our of the RES consoles – make it so they don’t need to be in the console

The automation underlying this has been evolving and is now moving to Decision Services and the Decision Governance Framework as well as working only in Decision Center. UrbanCode is used to manage the deployment automation, accessing the Decision Center and other ODM APIs, storing the compiled artifacts and managing the solution. State Farm originally built this themselves but newer versions have UrbanCode plugins.

State Farm really wanted to manage the executable object model – the XOM – so they could make sure the XOMs needed by RuleApps were deployed and available. Newer versions of IBM ODM allow you to embed the XOM in the RuleApp deployment so it is self-contained and not dependent on the correct (separate) deployment of the XOM.

End to end traceability is the end goal. Baselining the RuleApp means you know which rules and rule versions are in the RuleApp. In theory all you need to know is the ruleset that executed and the baseline of the RuleApp deployed – this tells you exactly which rules you executed. Decision Center tracks who deployed which RuleApp to where and when, linking the deployment to the RuleApp baseline. But to get this detail to the Ruleset level you need to add an interceptor to add the baseline details to each ruleset.

Versioning is critical to this traceability. An intentional versioning scheme is a must and deployment is done by replacing the old deployment explicitly so that version numbers are managed centrally. State Farm embeds version information in names. Most users just deploy asking for latest version, and this works automatically, but this explicit approach gives control and options for using named versions when that is needed.

Lots of very geeky ODM info!

Ginni Rometty kicked off day two of the IBM Interconnect conference, with a pitch that cloud is changing IT, business and indeed society. Cloud, and the IBM Cloud in particular, she says will allow a new generation of business and change the world. Cloud is already 17% of IBM’s business and clearly Ginni sees this growing rapidly. Three elements drive this:

  • IBM Cloud is Enterprise Strong
    Strong public infrastructure, industrialized hybrid, enterprise strong, choices and consistency, secure etc etc. Plus an industry focus to support specific regulatory frameworks, APIs etc. Need a pipeline of innovation too – IBM’s investing in blockchain, quantum and more to ensure their cloud continues to have innovative capabilities.
  • IBM Cloud is Data First
    The value of data, she says, is in the insights it generates for you – not democratizing and sharing your data, but letting you use your own data to drive your own insights. This relies on data diversity (to make sure that public, private and licensed data of various types can be used togather) and data control (to protect the IP represented by your data).
  • IBM Cloud is Cognitive to the core
    Ginni feels that cognitive is going to drive the future and only companies adopting it will survive and thrive. Watson she says ranges from Machine Learning to AI to Cognitive. This has to be embedded in the cloud platform. And it needs new “senses” like an ability process images, listen to audio. Plus motion, sensors to provide “touch”. And Watson is being specialized and trained with data from industry leading sources.

AT&T came up next to talk about the impact of broadband, pervasive mobile connections and how this enables access to the kind of cloud IBM is developing. AT&T also think content matters, especially in mobile as more and more content is being consumed on the mobile device. The work IBM and AT&T is doing essentially allows a BYOD mobile device to connect to IBM Cloud assets as effectively and securely as an on-premise network. Plus they are doing a ton of work around IoT etc.

Mark Benioff of Salesforce.com came up next to talk about how IBM is integrating Watson with Salesforce. 5,000 companies are both IBM and Salesforce customers. The integration is between Salesforce’s Einstein and IBM’s Watson. Critically of course this is about data – bringing the data Watson can access like the Weather Company with the CRM data that Einstein works on. This ability to pull “back office” data and combine it with customer-facing data in the CRM environment allows for new customer-centric decisioning. Mark said that initially customers are focused on things like prioritization of calls or service issues, next best action. But he sees the potential for these AI technologies to really change how people work – enhancing the way people work – but this requires companies use these technologies appropriately.

H&R Block came up next to talk about how they are using Watson and cognitive to change the way they provide tax advice to customers. The Watson component drives a natural language interview, is trained on the tax code to spot deductions and then makes additional suggestions for future changes that will help. A second screen engages the client directly, enabling an integrated conversation around the data being gathered. Interestingly the staff using the software have found it engaging and feel like they are on the cutting edge, especially since they branded it with Watson. Watson, of course, is continuing to learn what works and what does not. They are looking into how to extend it from the in person interviews to the online business and to customer support.

Royal Bank of Canada came on stage to discuss the move to mobile – to a digital experience -in banking. All this requires a focus on the cloud, on building new capabilities on the cloud, to deliver the agility and pervasiveness that are needed. A microservices architecture creates “lego blocks” that can be easily integrated and deployed rapidly. This speeds development but it also changes the way things are built. And this takes more than just training, it requires certification, experiences that move them up the curve, ongoing commitment. This matters because mobile is on the verge of overtaking online banking as an interaction. Online banking used to be one release a year, now it (and the mobile apps) do at least 6.

Ginni wrapped up with a great conversation with Reshma Saujani, founder of Girls Who Code, and how IBM is helping them train a million girls to code. Very cool program and some great opportunities and ideas being generated. Lovely to hear some teens talking working with cloud, AI, APIs and all the rest. And very funny that they had to explain who IBM was to their friends 🙂

I am going to be at IBM InterConnect this week. I am speaking with Kaiser Permanente at 2pm on Monday – Pioneering Decision Services with Decision Modeling at Kaiser Permanente – so come by and here me and Renee speak about our successes with decision modeling with DMN (Decision Model and Notation), business rules and IBM Operational Decision Manager (ODM). I’ll have some copies of my books to give away and it’s a great story – well worth hearing.

Right after I will be easy to find at the meet an IBM Champion booth. Come by and ask me questions about rules, analytics, decision management, decision modeling or whatever! In general I will be around until early Wednesday morning and checking my email plus I will be blogging from the conference (here on JTonEDM) when I get a chance as well as tweeting @jamet123.

Come by and say hi!


The Decision Management Systems Platform Technologies Report began in early 2012 as a way to share our research and experience in building Decision Management Systems. Since then we have extended, updated and revised the report many times. This week we released the latest version – Version 8 – with a new, easier to use format. There is so much content in the report now than one great  big document no longer works. To make it easier to use we have now broken it down into a set of pieces. We have also added content on technology for modeling decisions, upgraded significantly the section on monitoring and improving decisions and given the whole thing a refresh.

The documents are:

  • Introducing Decision Management Systems
  • Use Cases for Decision Management Systems
  • Best Practices in Decision Management Systems
  • Five Key Capabilities
    • Managing Decision Logic With Business Rules
    • Embedding Predictive Analytics
    • Optimizing and Simulating Decisions
    • Monitoring and Improving Decisions
    • Modeling Decisions
  • Selecting Products for Building Decision Management Systems

The first three are an excellent introduction for business or technical readers, while the others are more focused on those who are selecting or using the technologies described. You can download them for free on the Decision Management Systems Platform Technologies Report page.

As always, don’t forget that we have extensive experience helping organizations like yours define, configure and implement Decision Management Systems that deliver on the value propositions described in the Report. Our clients are leading companies in insurance, banking, manufacturing, telecommunications, travel and leisure, health management, and retail. Contact us if we can help.

I caught up with the folks from Conductrics to learn about their 3.0 release (I have blogged about Conductrics before). Conductrics has been in the decision optimization business for many years now. At its core Conductrics is about assigning customers to experiences. They recently released 3.0 with some new features.

Conductrics Express is a point-and-click tool to help set up tests and personalized experiences for web sites. It’s a browser extension with lots of control. Historically Conductrics has been more API-centric. Some audiences really like a “headless” and API based platform but others want something more UI-based. In addition, quick tests or temporary projects are common and the new visual setup lets them quickly set something up. For instance, figuring out which customers get which experience sometimes requires some quick A/B testing or experimentation and there is no time to work with IT etc. Conductrics Express sits on top of the API so can be evolved and integrated with other API based projects.

To make it easier to use machine learning, the new version supports explicit induced rules. This gives you an interpretable AI as it converts complex optimization logic into easily digestible, human readable decision rules. Users can use it for audience discovery and can either have it drive the experience or just “listen” to see what it would have recommended. This engine does trial and error or A/B testing and as you collect data it builds a decision tree for audience segmentation.

One of the nice features of the engine it that it predicts likelihood of success for offers but also predicts how likely an option is the best one. This enables you to identify both those experiences that are clearly better than alternatives despite having a low chance of success as well as those that seem significantly better but where there is a high degree of uncertainty. This reflects the reality that additional targeting is lower value for some (sometimes there’s just not much difference between best and worst). This lets you see the marginal benefit of targeting (v picking A or B) etc. and allows you to see poorly served audiences.

The current version allows Inline creation of options, easy specification of goals and has a Javascript API that allows packaging of logic into a file that is locally available e.g. on mobile app. You can also group agents into a multivariate agent for reporting and create mutually exclusive agents to make for more sophisticated analyses. Users can also add rules to constrain or force experiences, use predictive analytics in assignment or randomly assign people to learn from experiments. A single recommendation can be the objective or a list of recommendations can be. All this can be tied together using flowlets that specify logic to tie agents together using adaptive or random logic. This allows for more complex decisions where making one implies another.

Finally, there is a new faster JavaScript API to the existing web service API and a variety of deployment options from hosted to on-premise.

You can get more information on Conductrics here.

I have trained a lot of practitioners in the Decision Model and Notation (DMN) – I am closing in on 1,000 decision modeling trainees now – and one of the interesting questions is always their motivation for using decision models. As I look back across all the folks I have trained, four motivations seem to bubble to the top:

  1. Too many analytic projects go awry and they need a way to frame requirements for analytic projects to tie analytics to business outcomes
  2. They struggle with a “big bucket of rules” in their Business Rules Management System (BRMS) and need a better way to capture, manage and structure those rules
  3. They need clarity and consistency in how they make (or want to make) decisions that are likely to be only partially automated
  4. They need an executable representation of decision-making

These motivations have three things in common:

  1. They need to visually communicate their intent to a wide group of stakeholders, some of whom did not build the model
  2. They need a broad group of people – business experts, business analysts, data scientists, programmers and others – to be able to collaborate on the model
  3. They have tried using documents (requirements, lists of rules, spreadsheets) before but have found them inadequate

Decision models built using the DMN notation and a decent approach (such as the one Jan and I describe in Real-World Decision Modeling with DMN) deliver. DMN Decision Requirements models are easy for a wide group of people to collaborate on and clearly communicate the structure of decision-making. Combined with either DMN Decision Logic models or executable rule artifacts in a BRMS, they manage decision logic effectively and are an executable representation of your decision-making.

Now, there are those that feel like the motivation for decision models is almost always a desire for an executable representation – that only a decision model that is complete and executable is really useful. While this may be true of BPMN tool vendors – who need executable DMN to make their process models go – it is far from true among decision modelers more generally. In fact in our training and consulting projects we see more projects where a decision requirements model alone is tremendously useful than anything else.

Among those who want execution there is a split also: Some want the model to be 100% executable – to be able to generate code from it. Others prefer (as I do personally) to link a business friendly logical model to executable elements in a more loosely coupled way. This allows the business to keep things in the requirements model that are not executable (such as how people should decide to consume the result or characterize an input) while IT can make some performance or platform-specific tweaks that don’t have to show up in the logical model. The cores are linked (most decisions in the model have an executable representation in a BRMS that is editable by the same people who edit the model) but not everything has to be.

Whether you like the “press the big red button” approach that generates code from a DMN model or a blended DMN + BRMS model for execution, never forget all the other use cases where clarity, collaboration, consistency and structure matter even though execution doesn’t. There is a wide range of valuable use cases for DMN – it’s not just about executable models.

Prompted by an interesting post by Bruce Silver called DMN and BPMN: Common Motivation, Different Outcome?

Forrester analyst Mike Gualtieri asks “if analytics does not lead to more informed decisions and more effective actions, then why do it at all?” Specifically in a great post What Exactly The Heck Are Prescriptive Analytics? he says (emphasis mine)

Prescriptive analytics is about using data and analytics to improve decisions and therefore the effectiveness of actions. Isn’t that what all analytics should be about? A hearty “yes” to that because, if analytics does not lead to more informed decisions and more effective actions, then why do it at all?
Enterprises must stop wasting time and money on unactionable analytics. These efforts don’t matter if the resulting analytics don’t lead to better insights and decisions that are specifically linked to measurable business outcomes.

Now he calls it Prescriptive Analytics and we call it Decision Management but we are aligned on the value proposition – operationalized analytic insight. I also really like his focus on the intersection between analytics  and business rules (decision logic). As he points out you can wrap business rules around analytics to make them actionable (starting with the analytic) and you can also start with the business rules and use predictive analytics to improve them. The combination of business rules and predictive analytics is the powerful one. And our experience shows that using a decision model, built using the Decision Model and Notation (DMN) approach, is a great way to show how you are going to use these technologies together to make better decisions and how those decisions are specifically linked to measurable business outcomes – metrics or objectives.

In fact I am in Asia this week presenting on a pilot that demonstrates exactly this. We have been developing a decision model and decisioning architecture, implementing that decision model in business rules and using analytics to inform those business rules as part of a pilot for a large enterprise out here. We are excited about the results as they show exactly what Mike – and I – have been saying: That a focus on decisions, tied to business outcomes, modeled and implemented with business rules and analytics leads to better business results.

If you would like to talk about a pilot or other help operationalizing your analytics, get in touch.

Thanks to my colleague Gagan Saxena for spotting this great post

I recently wrote three articles for KDnuggets on the potential for decision modeling in the context of the CRISP-DM methodology for analytic projects:

  • Four Problems in Using CRISP-DM and How To Fix Them
    CRISP-DM is the leading approach for managing data mining, predictive analytic and data science projects. CRISP-DM is effective but many analytic projects neglect key elements of the approach.
  • Bringing Business Clarity To CRISP-DM
    Many analytic projects fail to understand the business problem they are trying to solve. Correctly applying decision modeling in the Business Understanding phase of CRISP-DM brings clarity to the business problem.
  • Fixing Deployment and Iteration Problems in CRISP-DM
    Many analytic models are not deployed effectively into production while others are not maintained or updated. Applying decision modeling and decision management technology within CRISP-DM addresses this.

Check them out. And if you are interested in how one global leader in information technology is using decision modeling to bring clarity to its analytic and data science programs, check out this Leading Practices Brief from the International Institute for Analytics. We have found that a focus on decision modeling early really helps get and keep analytics projects on track and makes it much easier to operationalize the results.

Zbigniew Misiak posted a great set of tips from practitioners BPM Skills in 2017 – Hot or Not – with my contribution here.  This is a great way to get some quick advice from a wide range of practitioners and experts. As someone with a particular focus on decision management and decision modeling I was struck by the fact that there were 5 distinct recommendations for DMN and decision modeling/decision management (including, obviously, mine).

Some quick follow up points on my note:

  • Hot
    • Decision Modeling and the DMN standard
      • There’s rapidly growing interest from companies in using decision modeling and the Decision Model and Notation (DMN) standard
      • Lots of this interest is coming from business rules practitioners
      • But process modelers using BPMN are keen to use it too as modeling decisions separately simplifies their process models dramatically
      • And analytic/data science teams are seeing the potential for decision models in bringing clarity to their business problem
    • Predictive analytics (not just process analytics)
      • Data mining, data science, predictive analytics, machine learning etc are all hot technologies
      • Practitioners need to move beyond dashboards and process analytics to include these more powerful analytic concepts
      • A focus on decisions and decision modeling is central to finding the places where these analytics will make a difference
    • Declarative modeling
      • Business rules are declarative. So, at some level, are analytics. Process models are procedural
      • Declarative models like DMN allow you to describe decision making without overlaying the procedures we happen to follow today
  • Not
    • Modeling business rules outside the context of decisions
      • There’s no value to business rules that don’t describe how you make a decision
      • You can, of course, describe your data and your process in a rules-centric way and that’s fine
      • But don’t think there’s value in just having a big bucket of rules
    • Embedding business rules directly into processes
      • Decisions are where rules meet processes
      • Embedded rules as gateways or conditions simply makes your process messy and complex
      • Even embedding decision tables as separate tasks will only work for very simple scenarios
      • You need to model the decision and create a decision-making component, a decision service, for the process to use
    • Procedural documentation
      • Works fine for proceses but a procedural approach for documenting analytics or rules simply documents how you do this today, not what you need it to do tomorrow

Check out everyone else’s thoughts BPM Skills in 2017 – Hot or Not

Tom Davenport had a great article recently on Data Informed – “Printing Money” with Operational Machine Learning. His intro paragraph is great:

Organizations have made large investments in big data platforms, but many are struggling to realize business value. While most have anecdotal stories of insights that drive value, most still rely only upon storage cost savings when assessing platform benefits. At the same time, most organizations have treated machine learning and other cognitive technologies as “science projects” that don’t support key processes and don’t deliver substantial value.

This is exactly the problem I see – people have spent money on data infrastructure and analytics without any sense of what decision they might improve with them. By taking a data-first and technology-first approach these organizations have spent money without a clear sense of how they will show an ROI on this. He goes on to talk about how embedding these technologies into operational systems has really added value to an operational process – this, of course, is the essence of decision management as he points out. As he also points out this approach is well established, it’s just much more accessible and price-performant now than it used to be. It’s always been high ROI but it used to be high investment also – now its more practical to show an ROI on lower investments.

In the example he discusses in the article, the solution stack

…includes machine learning models to customize offers, an open-source solution for run-time decisioning, and a scoring service to match customers and offers

Tom goes on to identify the classic elements of a solution for this kind of problem:

  • A Decision Service
    This is literally a service that makes decisions, answers questions, for other services and processes. Identifying, developing and deploying decision services is absolutely core to success with these kinds of technology. The graphic on right shows how we think about Decision Services:

    • It runs on your standard platform to support your processes/event processing systems
    • It combines business rules, analytics (including machine learning) and cognitive as necessary to make the decision
  • A Learning Service
    This is what we call decision monitoring and improvement. You connect the Decision Service to your performance management environment so you can see how different decision choices affect business results. This increasingly includes the kind of automated learning Tom is talking about to improve the predictive power of the analytics and to teach your cognitive engine new concepts. But it can also involve human intervention to improve decision making.
  • Decision Management interface
    The reason for using business rules and BRMS in the Decision Service is to expose this kind of management environment to business owners.

We have seen this combination work over and over again at clients – mostly with human-driven learning to be fair as machine learning in this concept is still pretty new. Our experience is that one of the keys to success is a clear understanding of the decision-making involved and for that we use decision modeling. You can learn more about decision modeling from this white paper on framing analytic requirements, by reading this research brief (if you are a member of the International Institute for Analytics – the organization Tom founded) or by checking out these blog posts on the analytic value chain.

I have a lot more on how these decision management technologies work together in the Decision Management Systems Platform Technologies report which will be updated with more on machine learning and cognitive in the coming months.

BPTrendsReal-World Decision Modeling with DMN CoverPaul Harmon over on BPTrends interviewed Jan Purchase and I about our new book, Real-World Decision Modeling with DMN. The interview covers a pretty wide range of topics – a definition of Decision Modeling, the bottom line value of modeling a decision, the difference between decisions and rules and how decision modeling helps projects. We talk about some of our memorable experiences of applying decision modeling, our favorite best practices and the value in applying decision modeling and the Decision Model and Notation (DMN) standard alongside BPMN. We outline our method for applying decision modeling and DMN, and talk a little about the book – who it is aimed at and what it contains (Over 220 practical illustrations, 47 best practices, 13 common misconceptions to avoid, 12 patterns and approaches and 3 worked examples among other things). We end with some thoughts on how to get started.

Anyway, I hope you enjoy the interview over on the BPTrends site and, if you do, why not buy the book? Real-World Decision Modeling with DMN is available on amazon and more details are available on the Meghan-Kiffer site.