≡ Menu

Silvie Spreeuwenburg of LibRT came up after lunch to talk about a rules-based approach to traffic management. Traffic management is a rapidly changing area, of course, thanks to IoT and self-driving cars among others.

When one is considering traffic, there are many stakeholders. Not just the road user, also businesses reliant on road transport, safety authorities etc. The authorities have a set of traffic priorities (where they need good flow), they have delays and they have restrictions for safety or legal issues. They manage this today and expect to keep doing so, even as technology evolves.

To manage this they create lots of books about traffic management, incidents and other topics for each road. Each contains flow charts and instructions. This creates a lot of overlap so its important to separate problem definition from problem solution and to be specific – differentiate between things you must or may not do and those that are or are not actually possible.

The solution involves:

  • Policy about priority and traffic management norms
  • Identifying decision points, flow control points and segments in the road
  • Standard services – increase flow, decrease flow, reroute on a link in the network
  • Decisions to decide what the problem is, determine right thing to do, see if there’s anything conflicting, execute

The logic is all represented in decision tables. And applying the approach has successfully moved traffic to lower priority roads. Plus it fits very well with the way people work and connects changes in policies very directly to changes in behavior.

Marcia Gottgtroy from New Zealand tax presented on their lessons learned and planned development in decision management. They are moving from risk management to a business strategy, supported by analytical decision management. The initial focus was on building a decision management capability in the department and they initially focused on GST (sales tax) and it went very well, producing a decision service with proof of STP, operational efficiency very quickly. The service also had a learning loop based on the instrumentation of the service. They automated some of this (where the data was good) but did manual analysis elsewhere – not trying to over-automate nor wait for something perfect.

After this initial success, the next step is to focus on business strategy and get to decision management at an enterprise level. Hybrid and integrated solutions supported by a modern analytical culture driven by the overall strategy. Need to define a strategy, a data science framework, a methodology – all in the context of an experimental enterprise. They began to use decision modeling DMN – using decision requirements models to frame the problem improved the clarity, understanding, communication. And it documented this decision-making for the first time.

But then they had to stop as the success had caused the department to engage in a business transformation to replace and innovate everything! This has created a lot of uncertainty but also an opportunity to focus on their advanced analytic platform and the management of uncertainty. The next big shift is from decision management to decision optimization. Technology must be integrated, different approaches and an ability to experiment are key.

Nigel Crowther of IBM came up next to talk about business rules and Big Data. His interest is in combining Big Data platforms and AI with the transparency, agility and governance of business rules. Big Data teams tend to write scripts and code that is opaque, something business rules could really help with. Use cases for the combination include massive batches of decisions, simulations on large datasets and detect patterns in data lakes.

The combination uses a BRMS to manage the business rules, deploys a decision service and then runs a Map Job to fetch this and run it in parallel on a very large data set – distributing the rules to many nodes and distributing the data across these nodes so the rules can be run against them in parallel and very fast. The Hadoop dataset is stored on distributed nodes, each of which is then run through the rules in its own Map job before being reduced down to a single result set – bringing the rules to the data. This particular example uses flat data, about passengers on flights, and uses rules to identify the tiny number of “bad actors” among them. 20M passengers per day so it’s a real needle in a haystack problem. The batch process is used to simulate and back-test the rules and then the same rules are pushed into a live feed to make transactional decisions about specific passengers. So, for instance, a serious set up with 30 nodes, could scan 7B records (a year’s worth) in an hour and a half, 1.2M/second.

It’s also possible to use Big Data and analytic tools to analyze rules. Customers want, for instance, to simulate the impact of rule changes on large portfolios of customers. The rule logs of rules executed in a year, say, can also be analyzed quickly and effectively using a Big Data infrastructure.

Vijay Bandekar of InteliOps came up to talk about the digital economy and decision models to help companies face the challenges this economy creates. The digital economy is driven by the explosion of data and the parallel explosion in IoT devices. While this data is increasingly being stored but little if any is being effectively used. We need applications that can manage this data and take advantage of it because its just not possible for even the best human staff to cope – autonomous, learning, real-time decision-making systems are required. These systems require inferencing, reasoning and deductive decision models. While the algorithms work, it can be cumbersome to manage large rule bases. While machine learning approaches can come up with the rules, integrating these manually can be time consuming.

Architecturally, he says, most organizations focus on stateless decisioning with a database rather than a stateful working memory. Yet the stateful approach offers advantages in the era of fast moving, streaming data while also taking advantage of the rapidly increasing availability of massive amounts of cheap RAM. This requires agenda control and transparency, as well as effective caching and redundancy/restoration.

It’s also important to add learning models with both supervised and unsupervised learning engines to handle the increasing volumes of data. These learning models need to be injected into the streams of data, he argues, to make decisions as it arrives rather than being pointed at stored data. In addition, combinations of algorithms – ensembles – are increasingly essential given the variety of data and the value of different approaches in different scenarios.

The combination of delivers an adaptive decisions framework for real-time decisions. It uses stateful decision agents based on business rules and continuous learning using ensembles of analytic approaches on streaming data.

Last up is Tim Stephenson of Omny Link. His recent focus is on smaller companies and one of the key things about the new digital economy is the way in which this allows companies to punch above their weight. Small companies really need to drive leads to conclusion and manage customers effectively. CRM systems, even if they start free, can be complex and expensive to use. To unlock the value and respond appropriately faster to serve more customers you need to do a set of things well:

  • Have a consistent, published domain model to make data widely available across channels. For small businesses, this means a simple but extensible customer domain model e.g. contact, account etc.
  • Use APIs to support a wide variety of interfaces – contracts. This supports lots of UIs including future ones.
  • Workflow or process to seamlessly drive data through the organization and its partners
  • Consistent decision-making that enforces policy and ensures compliance with regulations

He walked through how these elements allow you to deal with core scenarios, like initial lead handling, so the company can manage leads and customers well. You need to use APIs to record well understood data, decide what to do and make sure you do what you decided to do.

The value of DMN (especially decision tables) allows you to get the business people to defined how they want to handle leads, how they want to make decisions. They can’t change the structure of the decisions, in his case, but they can tweak thresholds and categories, allowing them to focus and respond to changing conditions. And these decisions are deployed consistently across different workflows and different UIs – the same decision is made everywhere, presenting the standard answer to users no matter where they are working (a key value of separating decisions out formally as their own component). Using Decision Requirements Models to orchestrate the decision tables keeps them simpler and makes the whole thing more pluggable.

The payback for this has been clear. One user found that the time saved was about 75% but in addition, the improvement in response time ALSO means the company closes more work. Even small businesses can get an advantage from this kind of composable, consistent, repeatable, auditable, transparent decision automation.

And that’s a wrap. Next year’s Decision CAMP probably in Luxembourg in September and don’t forget all the slides are available on the Decision CAMP Schedule page.

Little bit of  a late start for me so I am starting with Geoffrey De Smet from Red Hat talking about constraint planning. He points out that some decisions cannot be easily solved with rules-based approaches – they can be described as decision (and as a DMN decision model in our experience) but not readily made with rules and decision tables only.  His key point is that different decision problems require different technology

  • Eligibility is a rules problem
  • License plate recognition is a neural network (analytic) problem
  • Roster scheduling is a constraint planning problem

And our experience is that you can do this decision by decision in a decision model too, making it easy to identify the right technology and to combine them.

He went into some detail on the difference between hard and soft constraints and on the interesting way in which the Red Hat planner leverages the Red Hat rules format and engine to handle constraint definition, score solutions etc. They support various approaches to planning too, allowing you to mix and match rules-based constraints and various algorithms for searching for a solution. The integration also allows for some incremental work, taking advantage of the standard rule engine features.

I wrote about some of the early work around Drools Planner back in 2011.

I went next, presenting on the role of decision models in analytic excellence

If this resonates with you, check out this International Institute for Analytics Case Study
I’ll be back with the rest of the day to wrap up.

Bastian Steinart of Signavio came up after the break. Like Jan and I, he focused on their experience with DMN on Decision Management projects and the need for additional concepts. Better support for handling lists and sets, handling iteration and multiplicity for instance is also something they find essential. They have developed some extensions to support these things and are actively working with the committee – to show their suggestions and to make sure they end up supporting the agreed 1.2 standard.

They have also done a lot of work turning decision modeling in DMN into Drools DRL – the executable rule syntax of Drools. This implies, of course, that DMN models can be turned into any rules-based language, we would strongly agree that DMN and business rules (and Business Rules Management Systems) are very compatible. From the point of view of a code generator like Signavio however, the ability to consume DMN XML generated from a model, is probably preferable. With support for DMN execution in Drools this becomes practical.

Denis Gagne introduced how elements of DMN can and perhaps should be applied in some other standards. He (like us) has seen organizations gradually pull things out of their systems because they have separate lifecycles – data, process, decision-making etc. Extracting this helps with the disjoint change cycles but also engaging business users in the evolution of operations and systems. Simpler, more agile, smarter operations.

In particular, Denis has been working with BPMN (Business Process Model and Notation), CMMN (Case Management Model and Notation) and DMN (Decision Model and Notation). All these standards help business and IT to collaborate, facilitate analysis and reuse, drive agreement and support a clear, unambiguous definition. BPMN and CMMN support different kinds of work context (from structured to unstructured) and DMN is relevant everywhere because good decisions are important at every levell in an organization.

Trisotech wants to integrate these approaches – they want to make sure DMN can be used to define decisions in BPMN and CMMN, add FEEL as an expression language to BPMN and CMMN, harmonize information items across the standards and manage contexts.

The three standards complement each other and have defined, easy to use, arms-length integration (process task invokes decision or case for example). Trisotech is working to allow expressions in BPMN and CMMN to be defined in FEEL, allowing them to be executable and allowing reuse of their FEEL editor. Simple expressions can then be written this way while more complex ones can be modeling in DMN and linked. Aligning the information models matters too, so it is clear which data element in the BPMN model is which data element in DMN Etc. All of this helps with execution but also helps align the standards by using a common expression language – BPMN and CMMN skipped this so reusing the DMN one is clearly a good idea.

Denis has done a lot of good thinking around the overlap of these standards and how to use them together without being too focused on unifying them. Harmonizing and finding integration patterns, yes, unifying no.

Alan Fish took us up to lunch by introducing Business Knowledge Models. Business Knowledge Models, BKMs, are for reuse of decision logic. Many people (including me) focus on BKMs for reuse and for reuse in implementation in particular. This implies BKMs are only useful for the decision logic level. Alan disagrees with this approach.

Alan’s original book (which started a lot of the discussion of decision modeling with requirements models) introduced knowledge areas and these became BKMs in DMN. BKMs in his mind allow reuse and implementation but this is not what they are for – they are for modeling business knowledge in his mind.

Businesses, he argues, are very well aware of their existing knowledge assets. They need to see how they fit in their decision-making, especially in a new decision making system. Decision Requirements Models in DMN are great at showing people where specific knowledge is used in decision-making. But Alan wants to encapsulate existing knowledge in BKMs and then link BKMs into these models. He argues you can show the functional scope in a decision using BKMs and that by itemizing and categorizing these BKMs.

Each BKM in this approach is a ruleset, table, score model or calculation. The complexity of these can be assessed and estimates/tracking managed. This is indeed how we do estimates too – we just use the decisions not BKMs in this way. He also sees BKMs as a natural unit of deployment. Again, we use decisions for this, though like Alan we use the decision requirements diagram to navigate to deployed and maintainable assets. He thinks that user access and intent do not align perfectly with decisions. He also makes the great point that BKMs are a way for companies to market their knowledge – to build and package their knowledge so that other folks can consume them.

The key difference is that he sees most decisions having multiple BKMs while we generally regard these as separate decisions not as separate BKMs supporting a single decision.

Jan Vanthienen came up after lunch – not to talk about decision tables for once, but to talk about process and decision integration. In particular, how can ensure consistency and prevent clashes. Testing, verification, validation are all good but the best way to obtain correct models is to AVOID incorrect ones! One way to do this, for instance, is to avoid inconsistency e.g. by using Unique decision tables in DMN.

Jan introduces a continuum of decision process integrations

  1. No decisions therefore no inconsistency
  2. Decisions embedded in BPMN – bad, but no inconsistency
  3. Process-decision as a local concern – a simple task call to the decision – this limits inconsistencies to data passing and to unhandled decision outcomes
  4. A more real integration – several decisions in the DMN decision model are invoked by different tasks. This creates more opportunities for inconsistencies – a task might invoke a decision before tasks that invoke sub-decisions or that decision.
  5. Plus of course, no process only a decision – which also has no consistency issues

In scenario 4 particularly there are some potential mismatches:

  • You can embed part of your decision in your process with gateways creating inconsistency
  • You can fail to handle all the outcomes if the idea is to act on the outcomes with a gateway
  • You need one of the DMN intermediate results in the process you need to make sure this is calculated from the same DMN model
  • Putting sub-decisions in the process just to calculate them creates an inconsistency with the process model
  • Process models should invoke decisions in an order or a way that creates potential inconsistency with the declarative nature of the decision model. They can be recalculated but many people will assume they are not, creating issues

Last session before my panel today was Gil Ronen talking about patterns in decision logic in modern technical architectures, specifically those going to be automated. His premise is that technical architectures need to be reimagined to include decision management and business logic as a first class component.

The established use case is one in which policy or regulations drive business logic that is packaged up and deployed as a business rules component. Traditional analytic approaches focused on driving insight into human decision-making. But today big data and machine learning are driving more real-time analytics – even streaming analytics – plus the API economy is changing the boundaries of decision-making.

Many technical architectures for these new technologies refer to business logic, though some do not. In general, though, they don’t treat logic and decision-making as a manageable asset. For instance:

  • in streaming analytic architectures, it might be shown only as functions
  • In Big Data architectures, there may be questions that the data can answer or as operators
  • In APIs, there’s no distinction between APIs that just provide data and those that display decision outcomes

They all vary but they consistently fail to explicitly identify and describe the decision-making in the architecture. This lowers visibility, allows IT to pretend it does not mean to manage decisions and fails to connect the decision-making of the business to the decision logic in the architecture. A common pattern or approach to representation and a set of core features to make the case to IT to include it in architectures:

  • Make deployed decisions (as code) a thing in either a simple architecture or perhaps within all the nodes in a distributed (Big Data) architecture
  • Identify Decision Services that run on decision execution server (perhaps)
  • Identify Decision Agents that run in streams
  • He also identified a container with a self-contained API but I think this is just a decision service

All correct problems and things that would help. This is clearly a challenge and has been for a decade,. Hopefully DMN will change this.

After yesterday’s pre-conference day on DMN, the main program started today. All the slide decks are all available on the DecisionCAMP site.

Edson Tirelli started things off with a session to demystify the DMN specification. DMN does not invent anything, he says, but takes some of these concepts and defines a common language to express them. To take advantage of it we need to implement it, develop interchange for it and drive adoption.

Edson developed a runtime for DMN that takes DMN XML and executes on the Drools engine. This takes interchange files from tools and executes the logic from those files. This drives his focus – he’s thinking about execution. He has a set of lessons learned from this

  1. You need level 3 conformance to generate code. He walked through the conformance levels – all have Decision Requirements Diagrams but the decision tables go from natural language (level 1), simple expressions only (level 2) to full expression language (level3). He pointed out that level 3 is not much more than level 2.
  2. Conformance levels do not reflect reality in that vendors do not comply neatly with the levels nor is there an outside way to verify the conformance. To help with this, Edson (and others) are working on a set of Test that are publicly available to help folks test their ability to develop and consume XML.
  3. Spaces in variable names are a challenge but necessary because users really want them in their object names. This is not as hard as people think and is really important.
  4. DMN Type system is not complete because there are some like lists or ranges that cannot be defined although they are allowed in the expression language.
  5. Some bugs in the spec, but the 1.2 revision is working on these
  6. Gert involved with the community – the specification is a technical document and subject matter experts and others in the community are very helpful

Jan Purchase and I spoke next, discussing three gaps in the specification that we see in the specification. Here’s our presentation and you can get more on our thinking in our book, Real-World Decision Modeling with DMN:

Bruce Silver came next to discuss the analysis of decision tables. DMN allows many things to be put into decision tables that are “bad” – not best practices – because the specification cannot contain methodology, because there are sometimes corner cases and because there are some disagreements, forcing the specification to allow both.

Bruce generally likes the standards restrictions on what can be in a decision table and has developed some code to check DMN tables to see how complete they might be.  While these restrictions are limiting they also allow for static analysis. He checks for completeness (gaps in logic for instance), compares hit policy with the rules to make sure the rules and hit policy match and to spot problems like masked rules (rules that look valid but are never going to execute due to the hit policy). It recommends collapsing rules that could be combined and makes other suggestions to improve clarity.

It also applies “normalization” based on the work of both Jan Vanthienen and some of the later work done for The Decision Model by von Halle and Goldberg. These are applied somewhat selectively as there are some that are very restrictive.

A clear approach to validating decision tables based on DMN – very similar to what BRMS vendors have been doing for years but nice to see if for DMN.

A break here so I’ll post this.

The first day at Decision CAMP 2017 is focused explicitly on the Decision Model and Notation (DMN) standard.

Alan Fish introduced the ongoing work on 1.2. He quickly summarizes the new features in 1.1 – such as text annotations and a formal definition of a decision service. Then he went through the new features, starting with those that are agreed:

  • Annotation columns can be added to a decision table
  • Restricted labels to force the use of an object name as a minimum
  • Fixed some bugs around empty sets, XML interchange etc.

In addition, several key topics are being worked on. These three issues have not been voted on yet but we are tracking to get these done:

  • Diagram Interchange based on the OMG standard approach for this so that every diagram can be interchanged precisely as well as the underlying model. Given how important multiple views are, this is a really important feature.
  • Context free grammar is under discussion as it has been hard to make names with spaces, operators etc. parsable. Likely to use quotes around names with spaces and escaping quotes in names etc.
  • Invocable Decision Services to allow a decision to be linked to a decision service instead of a BKM. This allows a more complex reusable package of logic. Using Decision Services as the definition allows packages to be defined and reused without forcing encapsulation. This creates several difficult issues to be resolved but we (the committee) is making progress.

Bruce Silver then facilitated a discussion on what people liked and disliked about DMN.

Likes

  • Eliciting requirements using decision modeling is more productive, more fun, more rapid than traditional approaches to eliciting requirements.
  • The use of explicit requirements helps with impact analysis, traceability at a business level.
  • Really brings together an industry across multiple vendors in a positive way – it’s great to have a standard, customers like the idea that vendors are compliant.
  • Ability to mix and match analytics, AI, rules, manual decision-making, machine learning, optimization and all other decision-making approaches in a single model.
  • FEEL has supporters and has some great features – especially for certain activities like defining decision tables, possible for non-technical users to be very precise.
  • Ability for business users to clearly express their problem and share this with others to get agreement, prompt discussion – and to do this at several levels.

Dislikes

  • Perhaps too many details too soon, too much of a focus on the meta model and the internals v what a user can see.
  • Sometimes the standard is too precise or limiting – not allowing decision tables and output to have different names, for instance.
  • Dislike some of the corner case features because they can get people into trouble.
  • Not really any good ways to show which kinds of technology or approach can be used in decision modeling – perhaps some icons.
  • FEEL has people who don’t like it too, but this is partly because its new and a change and because it perhaps lacks some of the patterns and capabilities needed. More examples would be good too.

Last week, Silicon Valley research firm Aragon Research cited Decision Management Solutions as a visual and business-friendly extension to digital business platforms and named us a 2017 Hot Vendor in Digital Business Platforms. We’re delighted about this and feel pretty strongly that this validates our vision of a federated digital decisioning platform as an essential ingredient in a company’s digital business strategy.

The report’s author, Jim Sinur, said:

Digital Business Platforms combine five major technical tributaries to create a cornerstone technology base that supports the changing nature of business, as well as the work that supports digital. Enterprises that are looking to manage a complex or rapidly changing set of rules that empower outcomes would benefit from decision management as offered by Decision Management Solutions, especially when combined with predictive or real-time analytics.

The report says that what makes us unique is that business people can represent their decisions in a friendly, visual and industry-standard model while managing the logic and analytics for these decisions across many implementation platforms. We’re working with clients to create “virtual decision hubs” that map the complexities of enterprise decision-making to the underlying technologies that deliver the decision logic, business rules, advanced analytics and AI needed to operationalize this decision-making across channels.

Click here to view the report.

Open Data Group is an analytic deployment company. The company was started over 10 years ago and has transitioned from consulting to a product company, applying their expertise in Data Science and IT to create an analytic engine, FastScore.

Successful analytics require organizational alignment (specifically between Data Science and IT) to create coordination of systems and business problem collaboration. In addition to understanding analytics, companies are trying to leverage new technologies and modernize their analytic approach. To address some of these challenges, Open Data Group have developed FastScore.

FastScore is designed to address various analytic deployment challenges to monetize analytic outcomes including:

  • Manual recoding and other complexity
  • Too slow to deploy analytic models (largely as a result)
  • Too many languages being used
  • IT and analytic teams are not on the same journey – analytic/data science teams care about iteration and exploration while IT cares about stable systems and control.

FastScore provides a repeatable, scalable process for deploying analytic workflows. Open Data Group see the model itself as the asset and emphasize that a model needs to be language and data neutral as well as deployed using micro-services (they are a Docker container) to be a valuable, and future proofed, asset.

FastScore is an analytic deployment environment that connects a wide range of analytic design environments to a wide range of business applications. It has several elements, all within a Docker container. It also includes a model abstraction (input and output AVRO schemas, an initialization and the math action) that allows models to be ingested from a wide variety of formats (including, Python, R, C, SAS, PFA) and a stream abstraction (input and output, AVRO schema in JSON, AVRO binary or text) to consume and produce a wide range of data (from streaming to traditional databases) using a standard lightweight contract for data exchange.

The FastScore Engine is a Docker container into which customers can load models for push button deployment. Input streams are then connected to provide data to the model and output streams to push results to the required business applications or downstream environment. Multiple models can be connected into an analytic pipeline within FastScore. Models can be predictive analytic models, feature generators or any other element of an analytic decision. Everything can be accessed through a REST endpoint, with model execution being handled automatically (selecting between runners for R, Python, Java, C for instance). Within the container is the stream processor that will enforce the input and output schemas and a set of sensors that allow model performance to be monitored, tested and debugged.

Besides the core engine, additional features include:

  • Model Deploy
    A plugin for Jupyter that integrates the engine with the Jupyter data science tool. Allows a data scientist using Jupyter to develop models and then check that they will be able to deploy them, generate the various files etc.
  • Model Manage
    Docker container that hooks into running instances of FastScore and provides a way to address and manage the schemas, streams and models that are deployed. Can be integrated with version control and configuration management tools.
  • Model Compare
    New in the 1.6 release, allows models to be identified as A/B or Champion/Challenger pairs and manage the run time execution of the models. Logs this data along with the rest of the data created.
  • Dashboard
    Shows running engines and Model Manage abstractions, changes and manages the run time bindings and abstractions, provides some charting of data including that generated by Model Compare etc. Uses the REST API so all of this could be done in another product, too.

Plus Command Line Interface and REST APIs for everything.

Because all of this is done within a Docker container, the product integrates with the Docker ecosystem for components such as systems monitoring and tuning. The Docker container, allows easy deployment to variety of cloud and on premise platforms and supports micro services orchestration.

FastScore allows an organization to create a reliable, systematic, scalable process for deploying and using all the analytic models developed by their analytic and data science teams – what might be called AnalyticOps, a “function” created to provide a centralized place to manage, monitor and manipulate enterprise analytics assets.

More information on FastScore.

 

Avola Decision is a decision model-based decisioning platform migrating from supporting the proprietary TDM (The Decision Model) approach to support for the DMN (Decision Model and Notation) open standard. I reviewed the previous product and since then the team has been working on a new product. The new Avola Decision is .Net based on the backend and is available on-premise or on Azure for the SaaS version (public or private clouds). The UI has been rewritten and is completely HTML and browser-based.

Customers begin at a landing page – a dashboard where shortcuts and information that are used regularly such as notifications or tasks can be displayed. Customers can have many projects within their environment and projects can be created by non-technical users. Projects are within a domain (and a domain may have many projects) and individual projects can be linked to dependent domains to bring in shared content if the owner of that content allows it. Multiple Decision Services can be defined for the project and different members can be added with different roles. A separate identity server supports two-factor identification and allows custom security approaches for specific customers.

Domains contain business concepts (sets of data elements) and projects. Users work within a project and its related domain(s). They have instant free-text search across the project that shows hits for the search as it is typed. Explicit tags are coming soon, allowing objects to be tagged and managed using these tags.

Data elements can be defined based on a set of allowed types. Specific data types can be constrained to a specific set of values (value lists) or precision. These can be used as a glossary for multiple data elements with a where-used capability to see which data elements are using the definition and which decisions use that data. Documents such as policies can be uploaded to create Knowledge Sources and additional ones can be created that point to websites etc.

The decisions in a model can be viewed, either just those exposed as decision services or all decisions. As before, the diagram is generated from the logic being defined behind the decisions. Plans exist to allow editing of the diagram directly but for now it is based on the executable logic behind it. The diagram is DMN-like, using input data nodes as well as the boxed list of attributes (combining both styles of data presentation). In addition, the decision nodes are divided up to see conditions, operands, and metadata from both data and sub-decisions. Users can zoom in and out, restrict the number of levels being viewed, see the layer “above” a decision – the decisions that require it etc. Future versions will allow the user to hide the data elements, knowledge sources etc.

Behind each decision is decision logic, currently only as a decision table. Other DMN representations will be coming soon as will multiple action columns but they plan to continue to use the TDM layout for decision tables as well as some of the decision table features in TDM but not yet defined in DMN. The decision table editor has been upgraded to support row and column movement, in-line editing and change highlighting. Rules can be cloned and edited and some checking is built in such as type conformance, range overlap/underlap. A formula builder is used for calculations and users can click through to follow inputs to their sources. Importing and exporting FEEL that defines decision logic is a future possibility also but they don’t plan to expose it as a standard way to edit logic in the tool.

Testing can be done at any point. Test data collections can be defined and used to test the development or any deployed version. One or many test collections can be run and the status of each collection and test is shown, making a quick visual check for success easy. An Excel template can be downloaded to bulk create test cases or they can be entered/edited individually. Tests can be viewed in terms of the rows that executed in the various decision tables and the table can be opened for editing. Impact analysis – seeing the impact of a proposed change in terms of overall results – is also planned.

Once logic is tested and confirmed, there is an approval cycle and deployment support as you would expect. Once this process starts, the whole service is packaged up and encapsulated so it cannot be impacted by changes e.g. to a shared value list.

Executions store the version of the model used, the outcome of the decision invoked as well as the results of each sub-decision as well as the data used to create it. This information is available for reporting and analysis.

Future add-ons will allow the data defined to be used to define web forms or surveys to capture the data needed.

More information at Avola Decision.

When the famous nerd webcomic XKCD pokes fun at how you use technology, it’s probably time to try a different approach. Recently he posted on Machine Learning and took a swipe at the mindless way some people approach machine learning.  His characters discuss a “machine learning system” that involves pouring data into a big pile of linear algebra and stirring until you get the answers you want. Humorous though this is, it also represents a definite school of thought when it comes to advanced analytics – that more data and better algorithms is all you need. With enough data and algorithmic power there’s no need to think about the problem, no need to talk to the people who need the output, no need to do anything except “let the data speak”.

In our experience this approach has a number of problems:

  • Just because the data can tell you something, there’s no guarantee that the business cares about it, can use it or will use it for decision-making.
  • What the business needs – the analytic insight that they can use to make better decisions – is often not what your machine learning system/data science team think is most meaningful or important.
  • How accurate your analytics need to be, and how they need to be operationalized, in order to improve business decision-making cannot be determined from the data – only the people responsible for the decision can tell you this.
  • Most business decisions require policies and regulations to be applied too, not just the analytic insight from your data. Simply pushing data through machine learning or other analytic algorithms won’t tell you this.

In the end there is no substitute for knowing what the business problem is. In machine learning (predictive analytics, data mining, data science), this means:

  • Knowing what the business metrics are that show if you have succeeded.
  • Knowing what decision (or decisions) the business needs to make more accurately to achieve this.
  • Knowing how that decision is made today, what constrains or guides that decision, and how analytics might be used to improve the results.
  • Being able to place the analytic algorithms you develop into this decision-making context so they can be effectively used once you are done.

We have found on multiple projects that this is the biggest single problem – get the problem (decision) definition right and the odds of successful analytic projects (ones that actually improve business results) go way up. Decision discovery and modeling, especially using the Decision Model and Notation (DMN) standard, is tremendously effective at doing this. So much so that we do this as standard now on all our analytic projects.

But don’t just believe me – AllAnalytics identified this as the greatest problem in analytic projects  and research by the Economist Information Unit talked about the Broken Links In The Analytics Value Chain (you can find some posts on this over on our company blog – How To Fix The Broken Links In The Analytics Value Chain and Framing Analytics with Decision Modeling).

If you want to learn more, we  have a case study on bringing clarity to data science project as well as two briefs – Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model and Analytics Teams: 6 Questions to Ask Your Business Partner Before You Model – to show how a focus on decisions, and decision modeling, can really help. Or contact us and we can chat.

DataRobot is focused on automated machine learning and on helping customers build an AI driven business, especially by focusing on decisions that can be automated using machine learning and other AI technologies. DataRobot was founded in 2012 and currently has nearly 200 staff including 90+ data scientists. Since it was founded, over 200M models have been built on the DataRobot cloud.

DataRobot’s core value proposition is that they can speed the time to build and deploy custom machine learning models, deliver great accuracy “out of the box” and provide a simple UI for business analysts so they can leverage machine learning without being a data scientist. The technology can be used to make data scientists more productive as well as to increase the range of people who can solve data science problems.

DataRobot runs either on AWS or on a customer’s hardware. Modeling-ready datasets can be loaded from ODBC databases, Hadoop, URLs or local files – partnerships with companies like Alteryx support data preparation, blending etc. The software then automatically performs the kind of data transformations needed to make machine learning work – data cleansing, feature engineering needed for the various machine learning algorithms such as scaling and converting data to match the algorithms. It does not currently generate domain-specific potential features/characteristics from raw data, instead making it easy for data and business analysts to create them and feed them into the modeling environment. Once data is loaded, some basic descriptive statistics are loaded and the tool recommends a measurement approach (to select between algorithms) based on the kind of data/target.

DataRobot can apply a wide variety of machine learning algorithms to these datasets, for now almost exclusively supervised learning techniques where a specific target is selected by the user. Multiple algorithms are run and DataRobot partitions data automatically to keep holdout data for validation (to prevent overfitting), applies smart downsampling to improve the accuracy of algorithms and allows some other advanced parameters to be configured for specific kinds of data. Once started, DataRobot looks at target variable, dataset, characteristics, combinations of characteristics and selects a set of machine learning algorithms/configurations (blueprints) to run. These then get trained and more “workers” can be configured to speed the time to complete, essentially spinning up more capacity for a specific job.

As the algorithms complete, the results are displayed on a leader board based on the measurement approach selected. DataRobot speeds this process by running the blueprints initially only against a subset of the data and then running the top ones against the full dataset. Users who are data scientists can investigate the blueprints, see exactly the approach taken for the blueprint in terms of algorithm configuration, data transformations etc. Key drivers- the features that make the most difference – are identified and a set of reason codes generated for each entry in the dataset. Several other descriptive elements, such as word clouds for text analytics, are also generated to allow models to be investigated.

The tool also has a UI for non-technical users. This skips the display of the leader board and internal status information and displays just a summary of the best model with its confusion matrix, lift and key drivers. A word cloud for text fields and a point and click deployment of a scoring UI (for batch scoring of a data file or scoring a single hand-entered record) complete the process. More advanced users can interact with the same projects, allowing the full range of deployment and reuse of projects created this way.

Once a model is done, the best way to deploy them is to use the DataRobot API. A REST API end point is generated for each model and can be used to score a record. All the fields used in the sample are used to create the REST API and the results come back with the reason codes generated. Everything to do with modeling is also available through an API, allowing customers to build applications that re-build and monitor models. Users can also generate code for models but this is discouraged.

You can get more information on DataRobot at http://datarobot.com

The Rexer Data Science survey is one of the best and longest running polls of data mining, analytic and data science professionals. I regularly refer to it and blog about it. It’s time to take this year’s survey – and the survey is aimed at all analytic people, no matter whether they consider themselves to be Data Analysts, Predictive Modelers, Data Scientists, Data Miners, Statisticians, Machine Learning Specialists or any other type of analytic person. Highlights of the 2017 survey results will be unveiled at Predictive Analytics World – NY in October, 2017 and the full 2017 Survey summary report will be available for free download from the Rexer Analytics website near the end of 2017.

The survey should take approximately 20 minutes to complete. Your responses are completely confidential.

Direct Link to Start the Survey – Access Code:  M4JY4
Karl tells me it is OK to share this Access Code with your friends and colleagues as it can be used by multiple people. You can also get more survey information & FREE downloads of the 2007-2015 Survey Summary Reports from Rexer Analytics.

SAP BusinessObjects Predictive Analytics 3.1 is the current release of the SAP predictive analytic suite. Like most in the analytics space, SAP sees its clients struggling to make use of massive amounts of data that are newly available while facing ever increasing business expectations, faster business decision cycles and an analytical skill gap. SAP therefore is focused on predictive analytic capabilities that:

  • Produce accurate results in days not weeks
  • Deliver operationalization for machine learning at scale
  • Embed predictive analytics in business processes and applications

The predictive analytics suite consists then of four elements:

  • Data Manager for integrating and creating and reusing (potentially very wide) datasets
  • Automated Modeler, a wizard-driven modeling tool for predictive analytics
  • Predictive Composer, a more custom pipeline/workflow development tool for maximum control
  • Predictive Factory to operationalize all of this

These can access data from SAP HANA, SAP VORA, Hadoop/Spark, 3rd party databases and SAP HANA Cloud. And they can be embedded into SAP applications and other custom applications.

Four offerings package this up:

  • SAP BusinessObjects Predictive Analytics Suite for on-premise and for on cloud
  • SCP Predictive Services on cloud for embedding machine learning
  • SAP BusinessObjects Predictive Analytics for OEM/SAP application embedding

SAP is focused on speed, building models fast, but also on automating techniques. The assumption is that organizations need to manage hundreds or thousands of models and very wide data sets. Plus, for many SAP customers, SAP integration is obviously important. Finally, the suite is designed to support the whole analytic lifecycle.

The tools are moving to a new UI environment, replacing desktop tools with a browser-based environment. Predictive Factory was the first of these and more and more of the capabilities of the suite are being integrated, allowing Predictive Factory to be a single point of entry into the suite. As part of this integration and simplification, everything is being built to be effective with both SAP Hana and Hadoop. There is also an increasing focus on massive automation e.g. segmented modeling.

One of the most interesting features of the SAP BusinessObjects Predictive Analytics Suite is that there are two integrated perspectives – Automated Modeler and Predictive Composer. This allows data scientists and analytics professionals to build very custom models while also allowing less technical teams, or those with more projects to complete, to use the automation. All the models are stored and managed in Predictive Factory and Predictive Composer can be used to configure nodes for use in Automated Modeler. Predictive Factory also lets you create multiple projects across multiple servers etc. Existing models can be imported from previous tool versions or from PMML, new tasks (such as checking for data deviation or retraining models) can be created and scheduled to run asynchronously. Tasks can be monitored and managed, allowing large numbers of models to be created, supervised and updated.

The same automated algorithms can be accessed from the SAP BusinessObjects Cloud. Users can identify a dataset, identify something they are interested in and run automated modeling algorithms to see, for instance, what influences the data element of interest. This requires some understanding of the power and limitations of predictive analytics but no skill with the analytic technique itself. Data is presented along with some explanation and supporting text. The results can easily be integrated into stories being developed in the BI environment or applied to datasets. Over time, this capability will include all the capabilities of the on-premise solution.

Predictive Analytics Integrator allows these capabilities to be brought into SAP applications such as SAP Fraud Manager. Because SAP applications all site on SAP HANA, the Predictive Analytics Integrator is designed to make it easy to bring advanced analytics into the applications. Each application can develop a UI and use terminology that works for the application users while accessing all the underlying automation from the suite.

Predictive Analytics 3.2 in July will be the first release where the suite’s components are being integrated into the browser environment and the Predictive Composer name will be used. This release will not have 100% equivalence with the desktop install but will support the building and deployment of models using both the data scientist and automated tools.

You can get more information on the SAP BusinessObjects Predictive Analytics Suite here.

I recently worked with Tho Nguyen of Teradata on a white paper called Illuminate Dark Data for Deeper Insights

While organizations of all sizes across all industries are keen on becoming data-driven, most focus on only a fraction of the many types of available data. Not accessing a fuller spectrum of data, including those from “dark data”—emails, texts, images, photos, videos, and other documents—along with traditional data sources can limit an organization’s ability to gain a complete picture of their customers and operations, and exclude them from game-changing insights that improve business outcomes.

Dark data is defined by Gartner as “…the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes…”  (Gartner IT Glossary) and this paper discussed what makes data dark, what kinds of data go dark and how new technologies are illuminating this dark data.

You can register for the paper here.

If you want to talk about the role of dark data in your analytic and decision management systems, drop me a line.

Reltio Cloud is a modern data management Platform as a Service (PaaS) company focused on delivering data-driven applications, founded in 2011 by folks from Siperian which was acquired by Informatica. Unlike most data integration and MDM platforms, which are IT-focused, Reltio’s mission to make it possible for business and IT teams in enterprises to “Be Right Faster” by building data-driven enterprise apps that deliver reliable data, relevant insights and recommended actions. They compare these applications, based on broadly sourced, cross-functional data, with the traditional approach that delivers process-driven and siloed data. With data-driven applications contextual, analytical and operational data can all be brought together. This requires a reliable data foundation.

Reltio Cloud is a modern data management Platform as a Service (PaaS) and it includes:

  • Master Data Management as the core for delivering a foundation of reliable data
  • Predictive Analytics and Machine Learning through the inclusion of Apache Spark in the platform
  • A Graph Model allows for network and analysis integration across highly variable data sources
  • Big Data Scale and Performance so that transaction and newer data can be managed not just customer data
  • Workflow and collaboration capabilities to manage and curate data
  • Data as a Service is core to the platform so that third party data services can be easily integrated.

The graph schema is key to Reltio, allowing them to store both entities and their relationships in a semantically rich way. Data is stored in a combination of Apache Cassandra, graph technology, and in-memory structures such as Elastic. It offers an extensible structure for an organizations entities and relationships. The Reltio cloud collects data from multiple sources, matches, merges and relates them to create these relationship graphs and these graphics then underpin the data-driven applications being developed.

Reltio Insights shares objects (built from the profile and transaction data) with Reltio Cloud and analytics environments like Spark (either the Reltio platform or a customer’s own) to create analytic insights. These insights then get integrated with the master data so that these can be made available to data-driven applications. Reltio Insights is designed to rapidly provision master and transactional data into a Spark, environment. The resulting analytic insights are available throughout the Reltio environment, added to the data e.g. a customer’s churn propensity becomes an attribute of the customer profile.

The applications themselves can offer several different views – for instance, some users such as data stewards might see where the data came from and be able to interact with it to clean it up while others might only see the final, integrated view. A standard feature of the app is to visualize relationships, based on the underlying graph models. Some simple analysis, such as distribution of transactions by channel, can be easily included as can the results of more sophisticated analytics. Anything available in the Reltio data platform can be collaborated upon, managed and updated through data-driven operational applications. The data can then be used to drive analytical model development and provision the data to other operational applications. In addition, everything is tracked for audit and change purposes and the workflow engine can be used to manage requests for updates, changes etc.

Everything in the platform is available as HTML 5 widgets so that additional content like Google maps, can be easily embedded, and this means that Reltio content can also be easily embedded elsewhere. Many customers take advantage of this to mix and match Reltio content in other environments and vice versa. Similarly, all the data in Reltio Cloud is available from a REST API for use in all legacy operational and analytics systems.

You can get more information on Reltio here.

DecisionCAMP 2017 is coming up July 11-14, 2017 at Birkbeck College, University of London. This is going to be a great opportunity to learn about decision modeling, the Decision Model and Notation (DMN) standard and related topics. In fact the week is full of great things to do if you are in London or can make it there:

You can register for DecisionCAMP here.

One of our clients was presenting recently at a TDWI conference and was picked up on TechTarget – Analytics teams give data science applications real scientific rigor. It’s a great article with some good tips about using a repeatable methodology like CRISP-DM, especially when combined with decision modeling as a way to capture business understanding and drive collaboration (see this post too on Helping your analytic projects succeed with decision modeling and CRISP-DM). As Anu Miller of Cisco put it

We ask each other all the time, ‘What business decision are you looking to support?’

This focus on method and on business decisions also helps bring teamwork across the business/data science team divide too. As she went on to say

Those things almost force you to be collaborative. There are no unicorns on our team. We have to work together.

All good advice. If you live in the Bay Area, you can hear me talk about some of the key aspects of this approach at the Global Big Data Conference when I talk about ‘Don’t Apply Big Data Analytics To The Wrong Problem: Put Decisions First’. If you don’t live locally, check out this case study: Bringing Clarity to Data Science Projects with Decision Modeling.

And remember, if you would like to talk about improving your data science approach or other help operationalizing your analytics, get in touch.

Equifax has been expanding beyond credit bureau data in recent years by providing better access to a broad range of their own fraud, employment, wealth, commercial and alternative data sources as well as 3rd party data to position themselves as an insights company. As part of this focus, their Decision Management platform, InterConnect, was rebuilt from scratch as a foundation for cloud-centric and multinational decisioning applications. InterConnect is designed to support a broad range of decision management solutions, with an initial focus on customer acquisition.

InterConnect is designed to be a secure cloud-based decision management platform to define and execute decision policies at the front line. It is focused on delivering robust data, powerful decisioning and streamlined technology.

There are four main decisioning tools in the platform

  • Insight Gateway
    Streamlined transactional access to diverse data sources.
  • Attribute Navigator
    To manage and deploy the data catalog and derived attributes.
  • Model Integration tool
    A single tool to integrate, audit and deploy predictive analytic models into production.
  • Rules Editor
    A rules management environment for creating, testing and optimizing business rules

These four decisioning tools are presented in a common decision portal that is role-based, so only selected elements are exposed to users. This portal is gradually becoming the presentation layer for all Equifax services.

Insight Gateway gives real-time transactional access to data sources. This includes many standard data sources (such as Equifax’s own credit bureau data) as well as specific local data sources developed in each country  Insight Gateway uses a microservices architecture, JSON and self-description to make integration flexible and visual. It is supported by a Data Provisioning Tool that allows for discovery and conditional orchestration/integration.

Attribute Navigator allows users to create a catalog, define the attributes, test and deploy. It supports the definition of custom attributes against multi-bureau data, third party data or customer data. Customer data may be hosted by Equifax or can be sent on each individual transaction. The environment supports testing and auditing of attributes.

Model Integration Tool lets teams integrate, audit and deploy predictive analytic models. It supports scorecards as well as decision trees and ensembles. It can guide users through the integration of SAS, SPSS and R models to generate PMML ready for execution in the platform as well as a specification document for compliance and governance. This generated PMML, or other PMML models, can be executed in the Interconnect platform using the well-known Zementis engines (both ADAPA for cloud deployment and the Zementis Hadoop plugin – reviewed here).

Rules Editor is based on the modern rules platform provided by Sparkling Logic – SMARTS (most recently reviewed here). This provides a data-driven rule editor with authoring, visualization, testing and simulation all in one interface. The rule authoring environment supports cascading and inheritance, rule flow management, champion/challenger, trace execution and reports on key performance metrics.

Configuration of the four services for individual customer requirements and solution orchestration at runtime is delivered by Equifax’s professional services. InterConnect can be custom-designed or accessed as a pure platform utilizing all or individual service as needed. It is available in the US, Canada, UK, Spain, Peru, and Chile. Argentina, Paraguay, Uruguay, Mexico, Australia and India are expected to be added in the future.

You can get more details on the platform here.

Humana presented at InterConnect 2017 on their use of business rules on z/OS. Humana is a 3M member health insurer and a user of IBM Operational Decision Manager (ODM), IBM’s Business Rules Management System and has been focusing on using it to modernize some of their key mainframe systems – something that Humana is focusing on as part of its efforts to reuse existing assets. ODM runs on IBM z/OS for batch, CICS, standalone rules or WAS on z/OS, allowing them to run business rules on their mainframe systems. Using ODM allows Humana to reuse these assets while also transforming their development approach to be more responsive, more aligned with the business and more consistent to ensure compliance and manage risk.

Humana uses ODM for Medicare, Claims, Enrollment and Dynamic forms:

  • Humana has 700 Medicare plans that have to be reviewed for CMS compliance. A .Net application integrated with the decision service answers the CMS questions with an SLA of 2 seconds. The environment allows the business to manage the 1,700 rules in the application using ODM Decision Center. This improves the change cycle from months to weeks.
  • Claims processing handles multiple procedure payments and member cost sharing, for instance. Run as Cobol CICS and batch systems with 500+ rules and decision tables. 3.5M ruleset executions daily. Manual rules that could not be coded in COBOL now in ODM, increasing the rate of STP and driving savings. Savings in first week exceeded cost of development!
  • Enrollment for large and small group – about 30+ rule applications to reduce enrollment from a week to real time.
  • Dynamic forms is for authorization, generating custom questionnaires dynamically. 70+ questionnaires can now be developed and tested by the business. Complete audit trail and ability to make rapid changes have been key.

Architecturally

  • Humana runs ODM on z/OS.
  • Rule Designer (IDE) is used develop the vocabulary, templates, rules etc. This is tested and then pushed to Decision Center for ongoing business user management.
  • Decision Center is used across all environments. This allows business user engagement in a thin client environment and can be synchronized with the technical environment. Decision Center supports deployment, versioning etc and becomes the single source for rules. The support for testing and simulation are key and very popular with business users. They use both the Enterprise and Business Console, though the Business Console is the standard go-forward environment. All this runs on Z Linux, letting them take advantage of the integration of Z and DB2, the power of Z etc.
  • Decision Center is used to deploy Decision Services to the various environments Humana used – z/OS,Linux on Z etc.
  • The Rule Execution Server Console is used to monitor executing rules, trace rule execution and manage the deployed services.
  • They take advantage of the Java/DB2/z integration and performance tuning to maximized the performance of their rule execution. They mix and match decision services deployed for interactive or batch services, integration with COBOL or CICS etc etc. Lots of options for integrating the decision services into the mainframe environment.

Moving forward they are looking at some changes for high availability workload management as well as embedded batch for improved performance. Plus they want to complete the move to proper Decision Services from traditional rule applications.

Overall ODM on z/OS has delivered several benefits:

  • Cost savings
  • Improved time to market and business engagement
  • Single source of rules
  • Incremental adoption

State Farm presented at IBM InterConnect 2017 on their challenges with large scale deployment of IBM Operational Decision Manager (ODM) – IBM’s Business Rules Management System. State Farm has a set of specific things it wants out of its BRMS:

  • Well defined artifact lifecycle for auditing
  • Rigorous deployment process support for confidentiality and consistency
  • Self-service so authorized users can work 24×7

This is a pretty technical topic as State Farm is a large scale user of rules and IBM ODM. They have >500 rule projects with rulesets that vary from 100 rules to 10,000, some invoked every few minutes to very large volume batch jobs. Some of the decisions are trivial but others have big legal implications and must be 100% right. 45 different teams with 430 users of Decision Center are working on projects with over 80 deployment targets on Linux and Z/OS hosts.

They need RuleApps – the deployable units – to have well defined content, be accessible, controlled on need-to-know and governed appropriately for its criticality. Each RuleApp version is built once to ensure consistency and decouple deployment from the Decision Center editing environment. They are then promoted through the test and production servers. Its also important to manage the system users and business users appropriately.

Key precepts then:

  • RuleApp builds that are promoted for real use come from Decision Center
  • Well-formed RuleApp version baselines to track content
  • Self-service tooling to manage RuleApp, XOM, builds etc
  • Keep users our of the RES consoles – make it so they don’t need to be in the console

The automation underlying this has been evolving and is now moving to Decision Services and the Decision Governance Framework as well as working only in Decision Center. UrbanCode is used to manage the deployment automation, accessing the Decision Center and other ODM APIs, storing the compiled artifacts and managing the solution. State Farm originally built this themselves but newer versions have UrbanCode plugins.

State Farm really wanted to manage the executable object model – the XOM – so they could make sure the XOMs needed by RuleApps were deployed and available. Newer versions of IBM ODM allow you to embed the XOM in the RuleApp deployment so it is self-contained and not dependent on the correct (separate) deployment of the XOM.

End to end traceability is the end goal. Baselining the RuleApp means you know which rules and rule versions are in the RuleApp. In theory all you need to know is the ruleset that executed and the baseline of the RuleApp deployed – this tells you exactly which rules you executed. Decision Center tracks who deployed which RuleApp to where and when, linking the deployment to the RuleApp baseline. But to get this detail to the Ruleset level you need to add an interceptor to add the baseline details to each ruleset.

Versioning is critical to this traceability. An intentional versioning scheme is a must and deployment is done by replacing the old deployment explicitly so that version numbers are managed centrally. State Farm embeds version information in names. Most users just deploy asking for latest version, and this works automatically, but this explicit approach gives control and options for using named versions when that is needed.

Lots of very geeky ODM info!

Ginni Rometty kicked off day two of the IBM Interconnect conference, with a pitch that cloud is changing IT, business and indeed society. Cloud, and the IBM Cloud in particular, she says will allow a new generation of business and change the world. Cloud is already 17% of IBM’s business and clearly Ginni sees this growing rapidly. Three elements drive this:

  • IBM Cloud is Enterprise Strong
    Strong public infrastructure, industrialized hybrid, enterprise strong, choices and consistency, secure etc etc. Plus an industry focus to support specific regulatory frameworks, APIs etc. Need a pipeline of innovation too – IBM’s investing in blockchain, quantum and more to ensure their cloud continues to have innovative capabilities.
  • IBM Cloud is Data First
    The value of data, she says, is in the insights it generates for you – not democratizing and sharing your data, but letting you use your own data to drive your own insights. This relies on data diversity (to make sure that public, private and licensed data of various types can be used togather) and data control (to protect the IP represented by your data).
  • IBM Cloud is Cognitive to the core
    Ginni feels that cognitive is going to drive the future and only companies adopting it will survive and thrive. Watson she says ranges from Machine Learning to AI to Cognitive. This has to be embedded in the cloud platform. And it needs new “senses” like an ability process images, listen to audio. Plus motion, sensors to provide “touch”. And Watson is being specialized and trained with data from industry leading sources.

AT&T came up next to talk about the impact of broadband, pervasive mobile connections and how this enables access to the kind of cloud IBM is developing. AT&T also think content matters, especially in mobile as more and more content is being consumed on the mobile device. The work IBM and AT&T is doing essentially allows a BYOD mobile device to connect to IBM Cloud assets as effectively and securely as an on-premise network. Plus they are doing a ton of work around IoT etc.

Mark Benioff of Salesforce.com came up next to talk about how IBM is integrating Watson with Salesforce. 5,000 companies are both IBM and Salesforce customers. The integration is between Salesforce’s Einstein and IBM’s Watson. Critically of course this is about data – bringing the data Watson can access like the Weather Company with the CRM data that Einstein works on. This ability to pull “back office” data and combine it with customer-facing data in the CRM environment allows for new customer-centric decisioning. Mark said that initially customers are focused on things like prioritization of calls or service issues, next best action. But he sees the potential for these AI technologies to really change how people work – enhancing the way people work – but this requires companies use these technologies appropriately.

H&R Block came up next to talk about how they are using Watson and cognitive to change the way they provide tax advice to customers. The Watson component drives a natural language interview, is trained on the tax code to spot deductions and then makes additional suggestions for future changes that will help. A second screen engages the client directly, enabling an integrated conversation around the data being gathered. Interestingly the staff using the software have found it engaging and feel like they are on the cutting edge, especially since they branded it with Watson. Watson, of course, is continuing to learn what works and what does not. They are looking into how to extend it from the in person interviews to the online business and to customer support.

Royal Bank of Canada came on stage to discuss the move to mobile – to a digital experience -in banking. All this requires a focus on the cloud, on building new capabilities on the cloud, to deliver the agility and pervasiveness that are needed. A microservices architecture creates “lego blocks” that can be easily integrated and deployed rapidly. This speeds development but it also changes the way things are built. And this takes more than just training, it requires certification, experiences that move them up the curve, ongoing commitment. This matters because mobile is on the verge of overtaking online banking as an interaction. Online banking used to be one release a year, now it (and the mobile apps) do at least 6.

Ginni wrapped up with a great conversation with Reshma Saujani, founder of Girls Who Code, and how IBM is helping them train a million girls to code. Very cool program and some great opportunities and ideas being generated. Lovely to hear some teens talking working with cloud, AI, APIs and all the rest. And very funny that they had to explain who IBM was to their friends 🙂