Progress Corticon Business Rules Management System (BRMS) v5.2 was delivered in February and focused on key enhancements for natural language support, mobility, and Progress Software integration.  Since the first version nearly a decade ago, Corticon has focused on offering easy-to-use tools to express and manage decision making logic.  The Corticon business rules expression language is based on OMG’s OCL.  While this does not require programming skills to learn, it does require a few days of training for business analysts.  With v5.2, the Corticon rule language can now be mapped to natural language.  This enables business people, with no training in Corticon, to understand the business rules.  Additionally, Corticon now allows the business rules to be managed in natural language via the web or iOS (iPad, iPhone) interfaces. Additionally, Corticon v5.2 offers enterprise grade batch processing, enhanced XML messaging, and improved integration with Progress Savvion BPM and Apama BEP.

Beyond that, Progress Corticon will deliver a Web  based studio with global search and replace, as well as common repository technology across the Progress products and more integration that supports  a common object model across Apama, Corticon and Savvion.

From a decision management perspective, Progress Corticon provides the core decision logic management capabilities as well as some ability to solve constraint optimization problems.  The Progress RPM (Responsive Process Management) platform includes Progress Savvion Business Process Management (BPM) for executing decisions within the context of business processes, Progress Apama Business Event Processing (BEP) for monitoring events to make decisions at the earliest possible moment in time and Progress Control Tower to visual real-time information, including the results of decision processing.  Progress supports predictive analytics through third party integration.

In addition to providing RPM, Progress provides a number of vertical solution accelerators in Banking, Capital Markets, Supply Chain Management and Telecommunications. One of the solution accelerators sold to Banking and Telecom called Responsive Customer Engagement (RCE) illustrates the use of Progress RPM to deliver a next-best action marketing solution.  RCE monitors real-time event flows (including location and prior action) to determine what marketing offer to make a customer, and then sends the offer via mobile or web protocols.  Progress Control Tower allows typical dashboard views of data such as campaign performance, number of offers made etc. It also allows integration with business processes, allowing a new campaign process to be kicked off in context for instance. Screens in the process for data entry can be displayed and used to enter data. Approvals and completeness checks can be part of these processes and the Control Tower displays approval steps for the relevant approver. New elements, new portlets, can be added easily and live during operation. These could include, for instance, decision performance analytics or analytics on rule execution. The need for approval is driven by a decision task, using Corticon business rules.  Additionally, Corticon determines what offer to make the customer, executing a predictive model as a part of that decision to increase lift.  In this case the predictive model was created using KXEN, which predicts propensity to accept the offers based on prior similar customer behavior.  Corticon executes the predictive model while also considering other business rules such as availability, profitability and regulatory acceptability of the offer.

In summary, Corticon BRMS continues to be enhanced by Progress and is being combined with the Progress RPM suite. Progress is also continuing Corticon’s work on integrating predictive analytics with the BRMS. More information can be found at http://www.progress.com/corticon.

Progress is one of the vendors in our Decision Management Systems Platform Technology report.

Skytree is singularly focused on advanced analytics, machine learning on massive datasets. They have been in development for several years and are based in Silicon Valley (with engineering teams there and in Atlanta) and the product was officially launched in 2012. They believe that machine learning will be at the core of solutions for big data (solutions like Google AdWords or Netflix recommendations are based around machine learning techniques) and their product is designed to make scalable, general purpose machine learning infrastructure readily available. Skytree is positioning itself at the intersection of large data scale (data warehousing, Hadoop, storage appliances, etc.) and advanced analytics (Statistical packages and predictive analytic workbenches). In particular their focus is on pattern recognition, predictive models and data mining and increasingly doing these things against streaming data in a distributed computing environment. Most customers are using Skytree Server with other modeling tools but some are new to machine learning and advanced analytics and are only using Skytree Server.

Skytree Server has been designed from the ground up to perform machine learning efficiently. The standalone software version is already available and is cloud and Virtual Machine ready with standards-based data import and export. Full scale out distributed support (thousands of cores) and support for real-time/streaming data is coming later this year and is currently in private beta. The server delivers high performance for classic machine learning techniques such as K-Means clustering, SVM Classification and Nearest Neighbor. Comparing to open source solutions like WEKA and R they report 3-200x improvements over the more efficient of the two in each case (they can show up to 10,000x relative to the slower of the two) on just one machine. When the distributed version of the product is released, comparisons with distributed open source solutions will be available as well.

The product is purely API based and takes a data file or stream from any standard data source (RDBMS, NoSQL, HDFS, flat files). Machine Learning algorithms are then applied (to both variable identification and modeling) and the engine either outputs a file containing predictions, in batch scoring, or creates a model file that can be used to score individual records interactively in production mode. For those who want to build and continuously refine data the streaming engine gets a high degree of accuracy quickly as data streams in. Multiple streams can feed a single modeling routine (important when modeling in the cloud as it allows lots of end points to stream data to a single modeling routine). The streaming engine uses a TCP connection so it can be connected, for instance, to a listener on a drip-fed data warehouse  to analyze a data stream as the data warehouse updates.

Skytree is one of the vendors listed in our Decision Management Systems Platform Technologies report.

Sapiens has been around for over 25 years developing technology solutions around business rules and model-based development, specifically in financial services and insurance. Established in 1982 and NASDAQ traded, they have $100M in annual revenues post a couple of recent mergers. They have over 750 employees in US, Canada, UK, Japan, Australia, Belgium and Israel. Sapiens Americas headquarters operates from Cary, NC with additional offices in New Jersey and Toronto.  Their global customer is based largely in financial services and insurance.

DECISION powered by The Decision Model is a new product based on Sapiens’expertise in business rules architecture and management that was launched in 2011 to banking and mortgage clients in the US. Regulatory compliance, product definition and launch, terms of business management, data quality and more are typical projects. Business benefits from managing this decision logic include 50-75% reduction in time to implement new or changed products, services or regulations; business change without IT resources; enhanced traceability etc.

DECISION is a Business Decision Management System (a decision-centric business rules management system) based on The Decision Model invented by Barbara von Halle and Larry Goldberg (details here and the book is available on amazon.com). The product’s central repository is based around the artifacts of The Decision Model and all the graphical presentation conforms to it.

DECISION features completely browser-based editors with a rich graphical interface coupled with a Java Enterprise Edition backend built on standard components and a metadata repository for storing the information created. Graphical comparisons of the models, cross-referencing and traceability are available for Decision Models and the repository has a full set of user definable governance and workflow. Changes are tracked through a managed business change document that maintains traceability of everything being changed to the original policy or regulation document and section. A federated glossary, supporting different communities within the organization, is supported with a flexible data mapping layer.

While DECISION is essentially a repository of decision logic, it enables the implementation of the logic in business systems and business rules management systems through what it calls “Deployment adaptors.” Through these adaptors Sapiens currently supports implementing the business logic through its own rules engine known as eMerge, Red Hat’s Drools,IBM’s ILOG JRules, FICO’s Blaze and most other major BREs as well as a generic export capability. An SDK for deployment adaptors for other engines and execution environments is also available. The deployment of decision logic into executable systems is controlled by a release management system.

The web-based editors are all based on The Decision Model – the latest version that has evolved since the version published a couple of years ago in the book. The models have zoom, navigation panels, where used highlighting, jump to Rule Family, highlight uses of a Fact Type and more to manage large models. Any node can be selected and the user can navigate to the underlying tabular view of a Rule Family. This tabular editor is very Excel like and supporting Rule Families can easily be navigated within the tabular view or within the graphical view. As artifacts are navigated a pane displays all the cross-references and usage of the Facts or Rule Families being viewed. The Decision Models and Rule Families can also be viewed as a hierarchical model.

Underlying the Decision Models and Rule Families are a set of Fact Types. Fact Types are managed in a federated repository, allowing multiple glossaries to be defined in a hierarchy. This allows different communities within the organization to specialize and manage their own glossary in a controlled way while enabling reuse. Fact Types can be grouped into business concepts for navigation, have defined types and valid values, support synonyms and aliases etc. A target data mapping can be defined (table and column name say, to support the implementation of the logic into business systems and business rule management systems) and test values can be maintained separately from allowed values.

The software supports reuse and customization of business logic through Views and Communities of Decisions and Rule Families, a newer feature in The Decision Model. Views allow a specialized version of a decision to be developed. These can combine the base views of some Rule Families as well as specialized views of other Rule Families- the decision structure can be completely different if necessary or the variation could be minor and restricted to a single Rule Family. Individual rows within a Rule Family can be shared between views so that only new or changed rows need to be managed.  Associated decisions and rule families for a specific business context can be grouped and managed as a community.

Search and query tools run across the whole repository. The repository design enforces most of the structural requirements of The Decision Model and a validate button validates rows, Rule Families or Decisions against the declarative and integrity principles. A number of remaining checks are being implemented to validate all principles of The Decision Model. The repository also allows “Whiteboards” that represent a local copy of a set of objects so they can be edited and then merged back using tools for managing overlapping and contradictory changes.

The Business Change Document is at the heart of any change made using the tool. To make changes a business change must be identified, documented, and is defined through multiple tasks that can be assigned to people. Each task can then be linked to a specific workflow, defined in the repository and potentially custom to a specific community. These workflows can be completely customized and define the basic tasks, state transitions etc. Multiple workflows, for different levels of change for instance, can be defined and glossary/fact changes are managed separately from rule/decision changes. The change in state of a decision view is tightly coupled with the workflow, showing how the artifacts have to change as the workflow proceeds. Workflows can enforce segregation of duties by insisting on distinct users to ensure that people can’t approve their own work. Users see a work queue and manage their work from there. This can be integrated with an external workflow tool. These Business Change Documents and their associated workflow bring together all the work for a specific change to the model.

DECISION Version 3, which is in BETA and will be released to the public in early June 2012,  will also support test case management and generation. These can use the values in the Rule Families, the allowed values in the Fact Types or test data defined for the Fact Types (or all three). The different values are then combined into test cases. An expected result can be defined from the allowed results defined in the Rule Family and a comparison can be done. The results can be expanded to see what happened in each Rule Family, what messages were generated and can overlay the results on the graphical view of the model. Test results can be compared between versions also to see what changed in terms of test cases even if those differences are only in interim conclusions not overall outcome. Side by side views can be highlighted to show differences.

DECISION is cloud-ready and available on standard clouds such as Azure and Amazon.

Sapiens is one of the vendors listed in our Decision Management Systems Platform Technologies report.

While there are many machine learning packages out there, BigML is trying to make them usable by people with just a little experience – to improve the learning curve of machine learning techniques. At the same time they also address the scale challenge as Gigabytes of data override the traditional algorithms. Their product is therefore cloud-based and highly automated and supports a four stage process:

• Data sources
  Any data can be turned into a data source. Flat files can be dragged into the environment and other data can be loaded up using a robust API with bindings to many different languages. They offer an ability to bind to a URL and to Excel/databases as well as R and Ruby. With multiple ways to move data in to the cloud to be coming soon they allow a more incremental upload of data from multiple locations, helping address the issue of bandwidth when large amounts of data must be moved into the cloud from a single location.
• Datasets
  Once data sources are defined, a dataset can be created from a data source in a one click process. Today a dataset comes from a single data source but they are working on supporting data from several different sources in a single dataset. These datasets drive modeling and the automated process of converting data sources handles categorical fields, continuous variable distributions, automatic detection of categorical fields and so on. A dataset viewer allows the data to be explored online.
• Model
  The user selects the variable they want to predict and a one click process derives a model. Decision Trees are the only technique supported at launch with logistic regression, time series and naïve Bayes coming as well as some random tree techniques. These are home grown algorithms being developed specifically for the engine, although based on the industry standard approaches defined in the literature. The decision tree model can be viewed interactively and explored on the website including looking at the path to specific nodes or outcome for instance.
• Predictions
  Once you have a model you can pass a set of data in (using a form for testing or using the API) and get a prediction – a scoring API. You can also execute the model on a dataset for batch scoring. Users can also download a JSON document that describes the model and this document could, in theory, be used in a local implementation. They have what they describe as “a PMML compatible approach” and have given some thought to implementing a convertor.

BigML is invite-only at this point but have about 80 users so far. They are targeting developers who want to integrate analytics into their web-based application as well as those interested in using analytics but who lack a technical background in machine learning. Pricing is based on credits that get consumed by data uploaded, model create computation and predictions.

BigML is one of the vendors listed in our Decision Management Systems Platform Technologies report.

High-Performance Analytics from SAS consists of SAS Grid Computing, SAS In-Database Analytics and SAS In-Memory Analytics. The latter component has a new addition in the form of SAS High-Performance Analytics (SAS HPA), which was announced in Dec. 2011. SAS HPA is appliance-ready software that uses hardware from database partners (Teradata or EMC Greenplum) for high performance data exploration, model development and model deployment and scoring. The whole lifecycle from descriptive statistics to variable selection and modeling to model comparison and scoring is supported within these appliances.

Faster analytic model processing performance can make a real difference. For instance a large customer was spending 5 hours to develop a single algorithm (Neural Network) model each day for new customer acquisition business case. With SAS HPA they reduced the model processing time to 3 minutes, allowing a model to be iterated every 30 minutes or so while also using multiple modeling techniques. This improved their model lift from 1.6% to 2.5% – a potentially huge value for the enterprise with a large, diversified customer base.

One of the key elements of the SAS strategy is maximizing the value of existing investments. The infrastructure is designed to support users who want to write SAS analytic programs using SAS/STAT or use the graphical user interface of SAS Enterprise Miner to develop models interactively. This means existing single-threaded SAS code that uses SAS/Access for data read and runs on the client can be updated with a single proc name change (proc logistic to proc hplogistic). This makes the code multi-threaded, aware of the distributed computing environment, uses SAS/Access for parsing the data and then runs on the DBMS appliance in an in-memory fashion. In SAS Enterprise Miner there are simply new high-performance nodes for the analytic process flow that evoke in-memory processing using the database appliance and produce identical results for the rest of the process (scoring, deployment, model management, etc).

The EMC Greenplum and Teradata appliances for SAS HPA support data storage,  distributed execution of SAS code through the SAS embedded process engine as well as in-memory processing for analytics routines. As a result SAS HPA supports analytic data preparation, data exploration and model development (covering the most common data mining and predictive analytic routines). Obviously the in-memory routines also work on a SAS server too. In general SAS HPA requires data staging using data management processes.   The high-performance analytics procedures leverage in-memory computations for tremendous performance gains and run alongside the database to solve complex business problems on massive amounts of data.

Pervasive is best known for its data integration products but has recently been developing and releasing a series of products focused on analytics. RushAnalyzer is a combination of the KNIME data mining workbench (reviewed here) and Pervasive DataRush, a platform for parallelization and automatic scaling of data manipulation and analysis (reviewed here).

In the combined product, the base KNIME workbench has been extended for faster processing of larger data sets (big data) with a particular focus on use by analysts without any skills in parallelism or Hadoop programming.    Pervasive has added parallelized KNIME nodes that include data access, data preparation and analytic modeling routines. KNIME’s support for extension means that KNIME’s interface is still what you use to define the modeling process but these processes can use the DataRush nodes to access and process larger volumes of data, read/write Hadoop-based data and automatically take full advantage of multi core, multi processor servers and clusters (including operations on Amazon’s EMR).

The parallelized and distributable DataRush operators include:

-          I/O – JDBC, Delimited text, Log files, HDFS, HBase, Sparse data and PMML

-          Analytics – Association rules, Classifiers ( including Decision Trees, Naïve Bayes and SVM learners and predictors), Clustering (Recommenders and k-Means), Feature selection and Regression

-          Transformations – Aggregate, filter, manipulate

-          Data Profiling – Binning, percentiles, data quality metrics, pass/fail rules

-          Data Matching Fuzzy matching, encoding, clustering

DataRush itself is also extensible so users can add their own operators which can then be used just like other KNIME nodes.

All these nodes support multi-core and multi-processor environments and push processing to either the local desktop machine or to servers/blades that are available. If data is being read from or loaded into Hadoop clusters then the RushAnalyzer nodes execute within the nodes of the Hadoop cluster itself, pushing the function out to the Hadoop environment. While an all-Pervasive process executes fastest, the product can support mixed KNIME flows where not everything is a Pervasive node. One particular feature is that such a mixed environment can stream data between nodes without creating a local copy, something not available in the base KNIME product. The acceleration provided by this feature opens up more options to stage the data for multiple iterations on multiple models.

Pervasive is one of the vendors listed in our Decision Management Systems Platform Technologies report.

I have blogged about code effects’ Web Rule product before. This rule editor and execution environment supports both execution rules (that have an action to take) and evaluation rules (that just return true/false) as well as an IntelliSense/type ahead editor based on an XML object model. Rules in this product are closer to a ruleset in that many condition-action pairs or evaluations are in a single rule. The product is still open to storing rules anywhere – it simply generates an XML packet of the rule definitions (though the XML has been restructured and an XSD is now available to validate rule files before loading them).

The new version has a couple of enhancements. Most important is a new rule engine. The new version compiles the rules into a native .Net object, significantly improving performance. It can still evaluate rules against objects and now also supports filtering collections of in memory objects based on rules – determining which objects in a collection pass a rule.

From a user interface perspective they have added a Rule Bar that allows rapid access to a subset of the rules. This can be programmatically controlled so that a Rule Bar in a particular screen allows access only to certain rules, allows or disallows rule creation and editing etc.

Evaluation rules can be used to define patterns with their own names and description. Some or all of these can be added to the context menu dynamically. This allows the type-ahead editor to include them when it makes sense to do so.

Finally they have added ASP .Net MVC support as well as support for client-side Javascript/Ajax.

Rapid Insight was founded 10 years ago to develop tools that were easier for analysts to use to quickly extract meaning from data. Rapid Insight has been focused in higher education until recently and is expanding into fund raising and other areas. Rapid Insight Analytics is a pure data mining or predictive analytic workbench designed to support directed data mining using the maximum amount of automation.  To complete the analytic workbench, Rapid Insight Veera enables users to prepare analytic datasets.    Desktop versions are available for both products and a client-server version is available for Veera.

Veera is thus the first tool used in most analytics efforts. It is data agnostic and allows data to be pulled from multiple data sources of multiple types. Having established some connections to data sources (database, text files, Excel etc), the user can view the data structures in these sources and access them. Many jobs can then be managed, each containing a set of data sources and a visual workflow. These jobs can handle the usual kinds of tasks like filtering, cleansing, aggregation, de-duplication, merges, transformations etc. These jobs can output files suitable for the Rapid Insight Analytics workbench, write transformed data back to the database, score records using an existing model and so on.  When running against data in a database, Veera will do as much as it can in-database by generating SQL for filters, sorts, aggregations etc.

Rapid Insight Analytics’ projects start with an analytic dataset. The tool automatically assesses the variable types (continuous, binary etc) as well as missing values. It displays a wizard interface that shows the various stages of model building – variable statistics, data analysis, univariate and multivariate analysis, automated mining, variable creation, various kinds of analysis and then modeling, model comparison, what-if and reporting. Each tab within the wizard is a simple interface with many reporting and visualization tools at each level. A custom report can be assembled for a project from these various graphs – the graphs can be added to a set as they are viewed and that set can be turned into a report or exported to PPT.

The automated interface assesses the variables and finds those that are statistically related to the target variable. The user can then review each variable to see what kind of relationship, how strong a relationship etc. These graphs too can be added to the set for reporting.

The modeling tab allows logistic and OLS regression analysis. Predictive analytic models can be built automatically where each variable is assessed, transformed (bin values, use logs instead of raw values etc) and added to the model until no more predictive power is found. This can be done in a single automated step or the user can walk through the various steps manually, using features that suggest the next most suitable variable.

Models can be named and saved and then exported as a Rapid Insight scoring model file that can be used in a Veera workflow for scoring. Veera can also be used to set up processes to test model performance on an ongoing basis (i.e. monthly, weekly, daily) for model monitoring.

Rapid Insight is one of the vendors listed in our Decision Management Systems Platform Technologies report.

Rapid-I provides open source software for predictive analytics, data mining and text mining. Incorporated in 2006, they are based in Dortmund Germany and have been working on RapidMiner since 2001. They have over 35,000 production deployments and more than 400 customers in 40 countries. Banking and financial services is their largest market followed by Pharma and, interestingly, manufacturing. Customers include some large companies such as Siemens, Pepsico and Allianz. Their RapidAnalytics suite consists of RapidMiner and RapidReporting which offers traditional business intelligence reports and dashboard in addition to predictive analytics. Built on top of these are some solutions including RapidLab that is designed to be prescriptive (for instance to help people configure a machine based on its predicted/simulated behavior), RapidNet for network analysis and RapidSentilyzer to analyze web text for sentiment.

RapidMiner is a classic data mining workbench that allows a set of nodes to be linked to create data mining processes. Rapid-I think of RapidMiner as a process execution engine for the processes involved in data mining and analytics and provide a workbench to manage these processes as well as a server, web-based interfaces and an API. RapidMiner can be extended and has large numbers of extensions built by third parties. Rapid-I themselves also extended, for instance to handle Hadoop with a product called Radoop.

Key features of RapidMiner include:

• A GUI for analytics that handles more than 1,500 basic operations in the predictive analytic process with numerous extensions.
• Everything is considered a process so everything built in Rapid Miner can be extended and reused- there are no breaks between functions for ETL, data transformation and modeling for example.
• Supports in-database, streaming and Hadoop data
• Lots of automated analysis of problems and potential problems in the processes defined that are brought to the attention of the analyst
• Connectors for R, Weka and others
• A marketplace for extensions
• Standards support including PMML

RapidMiner provides a classic Windows UI that allows processes to be defined from a wide range of operators (both standard and extensions) using drag and drop. All programs are written in Java and hence can be executed on all major operating systems. Processes and other artifacts can be stored in a variety of repositories, both local and shared. Each process node can be inspected to examine its characteristics and meta data without having to execute it, allowing you to work with very large amounts of data without having to constantly process it as meta data propagates through the process. Any problems or inconsistencies (such as trying to apply a method that relies on numeric data to a text field) are flagged as the process is designed and the tool will suggest fixes such as adding a discretization node.

Process steps can also be added based on operator recommendations. These recommendations continually view the current process and suggest possible additional nodes such as cross-validation or champion/challenger testing. All the suggested nodes can be added from the short list of recommendations rather than having to go to the long list of operators, helping analysts rapidly build typical modeling processes.

Many operator nodes have a nested process that can be configured by drilling down into the node to see its process and all configuration of nodes is handled this way. Multiple sub processes can be defined for a node to handle things like training and testing. This decomposition can also be viewed as a tree or hierarchical view of the operator nodes. The view in the GUI can also be switched to a results view that allows all the interim results to be viewed and analyzed.

The operators allow for access to many data sources including all the major databases, large numbers of standard transformations and all the major modeling techniques. The GUI also provides a wide range of visualization and analysis tools to the view data being manipulated.

The server product RapidAnalytics shows recent changes to the repository and allows processes to be browsed as XML files (these files underlie the graphical view in the editor). Processes can be run from the server and can be scheduled. This can be configured using the server or from within the editing environment. The RapidAnalytics server offers several options for integrating the processes and models into other infrastructures. Any process can be exposed as a web service, either headless or with parameters. Processes can be configured to take parameters and this can be exposed through an API or entered in the editor or on the server. Results can be previewed through the server interface. This automation allows models to be built or re-built on a schedule and also allows a scoring process to be defined as a service for use in real-time scoring.

Processes also can be pushed into the database using an in-database extension that supports both DBMS-specific functions and a generic scoring operation that can be generated to execute as SQL on almost on any database. Models can also be exported as PMML. A stream mining extension allows for model re-tuning based on streaming data. Model monitoring can be implemented by building a second process that monitors the performance or behavior of a deployed model process.

Rapid-I is one of the vendors listed in our Decision Management Systems Platform Technologies report.

Our new report on Decision Management Systems Platform Technologies has been available for download for a while now and thousands of you have downloaded it. To make it even easier to access (the PDF does not require a registration even) we have made it available online in the main report webpage – just scroll down or use the table of contents to jump to the piece you are interested in. This version will be kept up to date as we enhance the report. Enjoy.

Don’t forget – bookmark decisionmanagementsolutions.com/decision-management-technology to see new versions and subscribe to our newsletter to get notifications of major new releases in this report.