Decisions, rulesets and business rules revisited

Some time back I wrote Here’s how decisions and rules relate (and how to manage them) and it seemed to me that this could do with being repeated with small updates.

One of the questions I get often is around how decisions and business rules relate. Business rules are, after all, one of the core technologies for building Decision Management Systems. As regular readers know, Decision Management Systems  are agile, analytic and adaptive and built using a three step process of decision discovery, decision service construction and ongoing decision analysis (all this is described in my most recent book in some detail and the three stages are described in webinar recordings linked to each of them).

Our first step, then, is Decision discovery. This involves defining a decision inventory for the area of the business on which we are working. First we need to find decision points (typically in our business processes but sometimes in our legacy applications or event processing scenarios). These are often clear, using decision words like determine if a customer is eligible for a benefit, validate the completeness of an invoice or choose which claims to Fast Track. Working with my clients I sometimes find that they are less clear and it takes some decision-centric thinking to find which bits of the process are really implementing decision making. But the end result is a set of decision points and matching decisions.

The key information about these decisions is often what question must be answered to make the decision – “is this particular customer eligible for free shipping?” say or “what information will be required from this person before we can process their loan application?” Some methodologies (and projects) stop here and then try to define rules. I find it helpful to decompose these decisions into the smaller decisions on which the bigger decision is dependent. I also find identifying the information and know-how being used to make the decision instructive. The decisions found in this process can themselves be analyzed and this recursive process results in diagrams like the one on the right. Here the decisions “Make retention offer” and “Price product” are both dependent on a decision to segment customers. Each is also dependent on various kinds of know how such a retention policy, risk models and pricing guidelines while information such as customer records and the product catalog are also required.

This decision dependency network shows how the decision works, helps us identify which decisions are candidates for automation and gives us a framework for linking decisions to processes, to events and to performance management and organizational structures also. Chapter 5 in my book has some discussion of this and the originator of the technique, Alan Fish, has a new book Knowledge Automation: How to Implement Decision Management in Business Processes that is now available (review soon – I wrote the foreword).

Decision Service construction is next. Decision services are services, generally stateless ones, that make decisions – that answer business questions for other services. Decision Services typically have no side effects so they can be called whenever they are needed without the caller worrying that something will change in the system. This means that database updates, event generation or other actions taken as a result of the decision are taken by the caller not by the Decision Service. This is not 100% true but works as a general rule. To work, Decision Services need to contain all the logic and algorithms necessary to make the decision correctly.

Decision Services:

• Support business processes by making the business decisions that allow a process to continue
• Support event processing systems by adding business decisions to event correlation decisions
• Allow crucial and high-maintenance parts of legacy enterprise applications to be externalized for reuse and agility
• Can be plugged into a variety of systems using Enterprise Service Bus approaches

All of these are shown in the graphic on the right (click for a larger version).

A decision service has some critical characteristics:

• Behavior understandable to the business
  After all we are talking about a “business decision” here so the business had better be able to verify exactly what is going on inside – what I call design transparency.
• Supports rapid iteration without disruption
  Business decisions change all the time so a decision service has to be both flexible and designed for this change. This is a key driver of the agile nature of Decision Management Systems.
• Integrates historical data
  Business decisions are increasingly made “by the numbers” with much reference to historical data. Decision Services need a similar ability to use historical data, and trends/insight extrapolated from it. This means embedded predictive analytic models that can be executed as part of the decision-making.
• Manages exceptions well
  Not only should it respond sensibly when it cannot decide, it should ensure that enough context is returned as to why it could not decide to assist a manual process.
• Must explain its execution
  Many decisions must demonstrate compliance or conformance with policy. Any decision service must be able to log exactly how it decided and that information must be accessible to non-technical users – it must have what I call execution transparency.

All of this drives the use of business rules, and a business rules management system, to manage the decision logic required.

So how then does a Decision Service relate to business rules? Take a look at the second graphic (also clickable for a larger view). Each Decision Service, and thus each decision requires a number of sets of rules – rule sets. These are coherent groupings of rules that can and should be used together. By and large individual rules are not that helpful for anything in business and a rule set or a collection of rules is required. Some decisions require a single rule set and some require many. These can be called decision models, rule families, rulesheets, decision tables, decision trees or just rulesets depending on the technology being used.

A single rule set decision might be something like deciding on the price of a product using a decision table or deciding what segment to put a customer in using a decision tree (decision tables and decision trees are graphical metaphors for a rule set). Rule sets can also be represented as rule sheets or simple lists – the best representation depends somewhat on the particular rules). What is important is that the rules in a rule set execute as a set and get reused as a set.

A multi-rule set decision typically has a decision flow (often called a rule flow by vendors) that lay out the steps involved in a decision, the branches and the loops and map the steps to specific rule sets. This is implied by the graphic as some decisions use more than one set of rules. This allows multiple decisions, for instance, to use customer segmentation rules as part of the decision without forcing a call to a decision service for customer segmentation. Because rule sets are coherent sets of rules on a single topic, they typically have an obvious business owner – an individual or a group. This allows a clear separation of rule management (by rule set) from rule execution (by decision) – there is not one rule maintenance environment for each decision but one for each set of rules. This allows decisions to use rules owned by multiple parts of the business, something that is important as legal, customer service, marketing and product could all easily have rules that contribute to a decision. Models too can be injected into these decisions in a similar fashion as shown.

As you can see these implementation objects implement the various decisions we identified in the discovery step and the rules within these steps might come from analyzing policy, from analytic modeling or from discussions with experts. The dependency network constrains the decision flow but the decision flow could contain additional design choices to make it work properly.

To summarize then:

• We separate decisions from processes, systems and events so they can be identified and managed explicitly
• We model and decompose those decisions to understand them and to identify how they can be automated
• We manage the rules for those decisions as coherent rule sets, each with a singular focus and common purpose
• The management of these rule sets is handled by those who are responsible for the source of those rules – legal documents, company policy, marketing strategy, call center expertise and so on

One of the consequences of this is that rule set changes are probably the crucial management item. Rule changes are going to be made within a rule set and often multiple rules in a rule set will have to change in response to new regulations, new policies or competitive changes. These rule set changes should be managed, tested, simulated and deployed. When they deploy they might alter the behavior of several decisions because they are reused in several and this is both to be expected and fine.

There can also be changes to decisions but these are more structural in nature – changing the flow of a decision or adding/removing rule sets to the approach taken. These may be triggered by changes to the rule sets but are more likely to be independent of them. In this case the decision changes but the rules and rule sets probably do not. Each decision is changed separately in this case but m experience is that these kind of changes are much less common than those involving rule sets.