I am speaking on “Driving Better Business Results with Decision Management Systems” on October 26th at 11:30am at IBM’s Information on Demand event.

Decision Management Systems are an exciting class of information systems. They help organizations manage risk, reduce fraud and maximize the value of every customer interaction. Decision Management Systems enhance existing business processes and legacy systems, adding value to what you already have. What’s more, the technology you need – business rules management systems, predictive analytic workbenches, and optimization systems – is proven, mature, and robust. This session will introduce the key principles of Decision Management Systems and outline three core steps for delivering always-on, customer-centric Decision Management Systems to maximize your operational effectiveness.

Register for IBM Information on Demand here.

Here is a recap of Decision Management product news for Q2 2011

Business Rules

• First Look Corticon 5
  A long established vendor in the business rules management space, Corticon describe themselves as focused on delivering better, faster decisions by automating business rules.
• First Look – OpenRules Decision Management System
  OpenRules allows business users to manage rules directly in Excel, OpenOffice or Google Docs (allowing collaborative rules management).
• First Look – Sparkling Logic
  A social, cloud-based environment designed to help companies find the best, most defined decision logic. 
• Update – New Wisdom RuleGuide
  RuleGuide manages information about decisions, rule families or rule sets, and a glossary of terms. It also provides analysis and governance for these objects. 

Data Mining and Predictive Analytics Platform

• Update – Angoss
  Release 7.5 of their data mining and predictive analytics software.
• First Look – Revolution and Netezza
  Recently announced partnership that supports both the creation of models in-database and the execution of the models in database.
• First Look – Yottamine
  Yottamine is focused on helping companies build predictive models.
• First Look – 11Ants Analytics
  11Ants Analytics is a spin-off out of the University of Waikato (the source of the WEKA project), and has commercialized technology for automating the production of predictive models.
• Update – Zementis
  Predictive decisioning platform, an update on recent announcements, partnerships and new features.
• IBM’s Big Data Platform and Decision Management
  IBM has recently announced a new strategy for bringing Big Data to the enterprise.

Optimization

• First Look – Drools Planner
  Drools Planner is a sub project of the Drools project that started 4 years ago as Drools Solver.
• First Look – OpenRules Rule Solver and Rule Learner
  Along with traditional Business Rules components, OpenRules includes two other important decision management components.

Decision Management

• First Look – Oracle Real-Time Decisions Manager
  Oracle’s core Decision Management product, RTD’s execution model is as a pure play Decision Service.

Business Process Management

• Update – Software AG
  An update from Software AG after they announced webMethods 8.2 and ARIS 7.2.
• IBM Business Process Manager 7.5
  The new Business Process Manager 7.5 brings together the WebSphere Lombardi Edition and WebSphere Process Server in a single go-forward product. See also IBM Websphere Update

Applications

• First Look – Activant DynaChange rules
  Activant (who recently agreed to be acquired and merged with Epicor) is an ERP provider focused on distribution and retail.
• First Look – Talent Analytics Advisor
  Solutions that provide relevant information to help companies anticipate the impact talent has on their business goals. 
• First Look – G Stat
  Advanced analytical and data mining solutions.
• Update – In2Clouds
  Update on added support for ensemble models, moved to allow private/hybrid cloud deployment and completed their service definition API.

Business Intelligence

• First Look – Information Builders
  Business intelligence (BI) and enterprise information integration/management (EIM). 

I recently published a First Look on the open source OpenRules Decision Management System. Along with traditional Business Rules components, OpenRules includes two other important decision management components:

• Rules Solver for solving optimization problems
• Rule Learner for predictive analytics.

OpenRules Rule Solver is based on Constraint Programming (CP) technology, that is typically used in scheduling, resource allocation or configuration problems with large numbers of possible solutions. CP solvers start with a set of predefined constrained objects, predefined constraints (arithmetic, logical, cardinality, all different, etc.) and various search algorithms. CP solvers do well when business rules run out of steam (because there are too many alternatives or there is a need to balance lots of “rules” against each other or find the solution that violates the smallest number of rules). CP solvers are successfully used for large combinatorial problems for years, but they have exploded recently with more offerings and a move towards standardization such as a CP Java API standard “JSR-331”.  CP standards should allow different solvers to be used interchangeably and simplify CP integration with decision management environments.

Rule Solver has been a component of OpenRules for a while but it is now compliant with JSR-331 and thus can utilize different CP solvers. Rule Solver uses the OpenRules rules syntax (decision tables) to define constraint satisfaction problems within an Excel-based editing environment. For scheduling problems, for instance, a simple table is defined that lists the Activities and their duration while a second table defines the Precedence Constraints between Activities.  There are tables for Resources with limited capacities, for Activity-Resource constraints like “Masonry requires Joe or Bill”. Rule Solver provides rule templates for many frequently used binary and global constraints allowing a non-technical user to work in Excel to define a business problem in terms of unknown variables and constraints on them.

The Rule Solver supports hard and soft rules/constraints. This allows it to solve over-constrained problems where there is no perfect solution that satisfies all constraints and the “least bad” solution must be found. The user defines hard and soft constraints with relative importance, limits of allowed violations, etc. and provides them as an input to the solver. The Rule Solver reads the information from these various Excel rules tables, creates the proper constraint satisfaction model on the fly, and executes the solver – producing a feasible or an optimal solution.  The Rule Solver is integrated with the rest of the OpenRules stack, by using the rules to generate a set of constraints, invoking the solver and then moving on to more rules that use the outcome of the solver for instance.  There is  documentation and a number of examples freely downloadable from http://openrules.com/download.htm.

Rule Learner is a component of the OpenRules BDMS that can generate business rules based on historical data.  It uses supervised machine learning. OpenRules provides a Rule Trainer that  automates the labeling of different data instances as “good” or “bad” – the crucial first step in supervised techniques. Rule Trainer is essentially a rule engine that executes “training rules” to generate training data sets. A business analyst uses Rule Trainer to define training rules in Excel rule tables or through a customizable web interface. Training rules specify criteria for selection of “bad” and “good” data instances and also allow further classification of “good” instances into different categories like “Very Good”, “Average”, “Not So Good”, etc. The Rule Trainer uses these training rules to extract instances from enterprise data sources using a highly configurable DB interface and to automatically label them – essentially building an analytic dataset.

The generated training records are then fed into the machine learning algorithms which will find the rules that predict how a record might be categorized using attributes that were not used to build the training data.  For instance, if the total amount of an order is used to generate the training data then the generated rules will not use total amount of the order (a potential “leak from the future”). The result is a set of rules that use other attributes to categorize your records similarly to your explicit categorization. This matters as you will typically have the extra details you used for training for only some records. The discovered rules will label all records. So, for instance, records you have not investigated or audited might be flagged as potentially fraudulent.

Each generated rule has an associated category and a score (hit rate) that reflects how predictive/certain the rule is. Weights can be provided by the user for each category and the combination of the weights and scores are used to rank new records. Thus, the generated rules can be applied against data that was not used for rule generation. Because the generated rules are presented in Excel tables in a human-readable (and machine-readable) format, a user may analyze, modify, activate, and deactivate the rules that will be used for actual classification of new data records.  As part of the verification process for these new rules the tool generates reports such as one that shows which rules fired on each of the records that are highly ranked. This allows experts to review why records were included evaluating them against the records with already known results.

The data mining algorithms are pluggable provided that they generate rules as an output from a training dataset – OpenRules default is to use different Weka classification algorithms such as RIPPER. Rule Learner is well-integrated with other OpenRules components. It includes a graphical interface that is based on OpenRules Dialog (described in the review of OpenRules) allowing creation of custom web interfaces for different rules discovery environments. While the discovered rules by default are generated as OpenRules decision tables and placed in different Excel files, the approach and its implementation are generic enough to use different formats and other rule engines. OpenRules Learner has been successfully used by such large organizations as the IRS.

Drools Planner is a sub project of the Drools project that started 4 years ago as Drools Solver. Drools Planner is focused on “NP complete” planning problems where a solution is hard to find, but easy to prove feasible once found. Examples include scheduling, bin packing and timetabling for instance. Drools Planner is particularly good at larger ones with lots of constraints, such as a staffing/roster problem where they are many roles and constraints as to who can be scheduled when. Drools Planner has a number of live projects and over 2,000 downloads of the 5.1 release (probably a lot more). It is not currently “productized”, meaning you can use it (under the open source Apache License) but you cannot buy “official” support from Red Hat yet (unlike for Drools BRMS for instance).

Drools Planner, like its competitors, considers both hard and soft constraints (though you can extend the Score interface to add extra levels of constraints). Hard constraints are those that must be met – for example all required shifts must be assigned to someone and no-one is assigned more than one shift per day. Soft constraints are those that can be violated but where violating them reduces the value of a solution. The solver will essentially try and find a solution that violates no hard constraints and minimizes the degree to which soft constraints are violated. Drools Planner is aimed at normal Java programmers who find themselves tasked with solving an optimization problem.

Constraints in Drools Planner are implemented using the Drools Rule Language (DRL) – the conditions of the rule are the constraint itself and the action of the rule is to lower the “score” of the solution. Thus solutions that break more constraints have a lower score. A score contains how many hard and soft constraints are violated. This approach means that the different constraints are clearly isolated and can have a weighted negative or positive impact on the solution’s score. The rule engine also ensures that if something changes during solution generation and a constraint is no longer violated then the score is amended appropriately and efficiently.

Drools Planner’s strength is to handle large solution sets and lots of constraints.  Drools Planner was used in a research contest 3 years ago with over 10^3023 solutions and 14 constraints (http://www.cs.qub.ac.uk/itc2007/) where it placed in the top 5 out of over 200 entries. The limit on the number of constraints it can handle has not yet been determined.

Drools Planner uses POJOs for input data and result data so it is easy to pull/push data from/to services, databases, jBPM etc. Developers can reuse Java methods (such as isHoliday(country, date) for instance) in constraints.

There are a number of interesting side effects of using the Drools rule language and integrating with the Drools rule engine. Using the forward chaining rules engine means that only potentially changed constraints must be re-evaluated: this “delta based score calculation” – rescoring the solution only based on the changed constraints – dramatically improves performance. Also, each performance improvement in the Drools engine usually improves the Planner speed. In addition, using Drools Guvnor, non technical users can be empowered to write constraints (today this is a bit of a hack but it is possible and will get easier). This would allow, for instance, a constraint to be exposed using standard Guvnor features so that business users could change the constraint in defined ways without getting into the code.

The solver itself uses local search algorithms – a search path is built by trying a set of possible changes, the one with the best result is then adopted as a new start point for another set of changes. To avoid getting stuck in local optima the local search engine has “tabus” – it has a solution tabu (won’t go to a solution it has seen before), a move tabu (if it has done something recently it won’t do it again yet) and a property tabu (if it has changed this value recently it won’t change it again soon). Simulated annealing is also supported (this is apparently quite hard to implement correctly). This algorithm initially allows fairly random changes to the solution and as it runs through solutions it gets “pickier” – only allowing better solutions. The randomness helps prevent the solver getting stuck in local optima while the decreasing tolerance of bad results causes it to drive to a solution. Of course you can add your own algorithms too, this being an open source project.

Here are a couple of good presentations on Drools Planner – http://www.slideshare.net/ge0ffrey/pushing-the-rule-engine-to-its-limits-with-drools-planner-parisjug-20101109/12, http://vimeo.com/9204264 and http://www.parleys.com/#id=1714&sl=1&st=5

The official Drools Planner page is here (http://www.jboss.org/drools/drools-planner). Obviously you can download Drools Planner from Jboss ( http://www.jboss.org/drools/downloads). You can get the sources of the examples from the download page or directly from github (https://github.com/droolsjbpm/drools-planner/tree/master/drools-planner-examples/src/main) and the manual at http://downloads.jboss.com/drools/docs/5.1.1.34858.FINAL/drools-planner/html_single/index.html. The next release, 5.2.0, is coming soon.