≡ Menu

Business rules, decision making and IT


When I blogged about Jim Sinur’s session “Business rules are king” at the Gartner BPM Summit I provoked a very thoughtful response from Tom Graves – On business rules. While Tom strongly agreed with the basic implication that organizations need discipline around business rules he identified three concerns, all of which seem to me to deserve a response.

  1. placing all the business-rules into an automated system will lead to a ‘fit and forget’ attitude unless there is a very strong emphasis on rule-maintenance – one of many ‘human factors’ that were forgotten about in BPR’s rush to ‘IT-ise’ all business processes
  2. identification and codification of business-rules assumes that the rules that can be derived from the people who run the existing processes are sufficient, invariant, accurate and complete – which, as early-generation knowledge-management also discovered, they rarely are…
  3. the viability of using automation for decision-making is dependent on the context – a fact of which frighteningly few IT-system designers seem to be aware

Let’s take these in the order Tom wrote them.

His first point is both true and absolutely crucial for successful adoption of decision management as an approach and of business rules as a technology. Indeed one of, if not the, key reason for adopting decision management and a business rules management system to manage the rules behind the decisions being managed is to ensure that the rules are not “fit and forget”. Unlike the alternative (coding something), the use of a business rules management system ensures that business experts can participate in maintaining the business rules in the system, that the business rules are clear and explicit not buried in code and that changes can be made, validated, verified and deployed quickly and easily. The value of adopting business rules technology and a decision management approach is much greater during maintenance and evolution than it is during development. I am a strong believer in business rules largely for this reason.

His second point is a little bit less on the money. Firstly, as noted above, the very ease with which rules can be found, changed and re-deployed when we find out we were wrong about the rules (or when the regulations change or whatever) is critical. Business rules people don’t assume that rules derived from experts are sufficient, invariant, accurate or complete. In fact we regularly supplement the rules they tell us with rules derived directly from the regulations they are supposed to be following and rules derived analytically from the data about what we did in the past and what worked or did not work. It is also off the mark because many business rules-based decision making systems do not attempt to handle 100% of the cases but 80% or 90% – the ones where we can determine the rules – and leave the others for a person to work. And again, remember the alternative. If the decision for which we are attempting to capture the rules is high volume, must be made in less than a second or delivered through an automated channel (as many decisions must be) then we must either capture these rules or write code. And, frankly, everything Tom says about rules is much worse when applied to code that the business experts can’t read!

Back to his third point and he is right again. In fact I am very explicit when identifying the kinds of rules that you should use the approach on – high volume, low individual value, operational execution decisions. In fact I think there is a sliding scale between decision support and decision management along which any given decision can be put to see how much automation is sensible (this is a concept Neil and I introduced in Smart (Enough) Systems)

As I continued in Tom’s post it seemed like a lot of his concerns came down to wondering about “using IT for all decision-making”.  He even accuses me, indirectly, desiring the automation of all decisions (something that is absolutely not true, as anyone who has read the book or this blog would know). I would respond to Tom, and anyone else worried about this, to go look at your business and see who makes the decisions now. High volume, low latency decisions are made by systems – by IT. They are made poorly. They are made the same for everyone (when they should be customer-centric). They are made following out of date regulations and policies (because it takes too long to change the system and anyway the business people don’t understand the system enough to tell IT what’s wrong). They are out of the business’ control, can’t be explained or updated and are generally not managed. Adopting business rules technology and managing decisions can improve this. Recommending this does not mean I think that “everything can be reduced to simple binary rules” or that you don’t need “real people” who “can handle decision-making in .. those contexts [where] there are no rules”. It just means that you should manage the operational decisions at the heart of your day to day business.


Comments on this entry are closed.

  • Tom Graves March 31, 2010, 12:23 am

    Hi James – many thanks indeed for the comments and response.

    First, a mea culpa: I hadn’t read your book, so I only responded to your original post in isolation from its broader context. And I also brought in perhaps too many assumptions of my own based on my unfortunate experiences with too many IT-obsessed ‘business-rules’ proponents who – unlike you – seem unable to grasp the inherent limitations of automation. So we’re much more in agreement than I’d thought, and from what you’ve said above, I would need to amend some of my critique.

    On points 1 and 2, I we seem essentially to agree: there are very real dangers in a ‘fix and forget’ approach – of which, as you say, probably the worst example is rules embedded unreadably in code – and we must provide some systematic means to identify and update the rules, algorithms, guidelines and principles used by the business for business decision-making. The latter is essential for proper business governance. But here ‘system’ needs to be understood in a broader sense, as a combination of information and the processes that act on those information-items – and, despite the claims of many would-be vendors of IT-based ‘business-rule systems’, much of that ‘system’ cannot and must not be ‘automated’. (And, as you say, the management of the automatable parts of that system must by accessible and understandable in business terms.)

    Yet perhaps the most important problem there is a knowledge-management dilemma that I first heard described by Dave Snowden: “people know more than they can say; and they can say more than they can write down”. Business-rules management is actually a sub-category of knowledge-management: and if we can’t write it down, we can’t embed it into an automatable ‘business-rules engine’. I’ve come across a very frequent tendency amongst IT-centric folks to assume that if something can’t be written down, it either doesn’t matter or doesn’t exist – both of which are extremely dangerous assumptions in the business-rules domain. We need to be absolutely systematic in reminding people at all times of the limitations of any system which relies on codified, explicit knowledge of this type.

    The other danger, which we could also note from Snowden’s ‘Cynefin’ framework, is that even the notion of ‘business rules‘ tends to break down in many business contexts. This is especially true in Snowden’s ‘unordered’ domains – the emergent or Complex, where rules can only be derived retrospectively, and will often change with each iteration, and the ‘market-of-one’ Chaotic, where by definition there are no discernible rules at all. Yet we still need consistent decision-making across all business-domains: a ‘business-rules’ (sic) model needs to be able to handle rules (Simple), algorithms (Complicated), pattern-based guidelines (Complex) and principles (Chaotic) as a single continuum of decision-making, otherwise we end up with something that hits an arbitrary yet probably undefined cut-off at which its assumptions will fail and other unspecified, unmanaged decision-making techniques must somehow take over. I now understand that you’re aware of such issues: what worries me is that far too many in the business-rules industry don’t seem to be aware of them at all.

    I do agree, and strongly, that a business-rules approach applies well in Simple-domain contexts, especially those which operate close to real-time: “high volume, must be made in less than a second or delivered through an automated channel”. (I think you’re being overly-optimistic about the scope for automatability, though: Sigurd Rinde suggests a typical figure closer to 30% than the “80% or 90%” you imply above.) It’s what happens beyond that Simple range that worries me.

    This also crosses over to point 3 above in another form, the question of how people develop the decision-making skills needed to handle the ‘unordered’ domains. As I wrote on my Sidewise blog, ‘Where have all the good skills gone?’, the usual skills-learning process is that people progress from competence at Simple to Complicated to Complex to Chaotic through a well-documented path that we might describe as Trainee, Apprentice, Journeyman and Master. The point is that each level of competence builds on the experiences developed in the previous level: we can’t simply jump in at the middle and expect to get reliable results. But that is exactly what happens when we shunt all Simple and, increasingly, Complicated decision-making onto ‘business-rule engines’ – if all lower-level decision-making is handled by machines, there is no means to learn the competencies for contexts that the machines can’t handle. The danger is that by piling complication on complication in IT-systems – that which Roger Sessions, for example, describes somewhat misleadingly as ‘complexity’ – we risk ending up with systems that are beyond anyone’s competence to manage. At the big-picture scale this was one of the major drivers for the current banking mess, but at a more mundane level we can see it in the way that motor-mechanics are now often reduced to checking computer-codes, without any means to know what’s actually going inside the engine.

    Much to discuss, of course – but my thanks for the conversation so far, anyway.

  • seamus walsh March 31, 2010, 8:58 am

    It sounds to me that the rules base approach might not be a good fit for everyone. Might I suggest a new look at this old problem (IT and business alignment)?

    I call it the “squawk” method;

    The squawk method looks at SG&A functional process and activities and follows content flows. Upon examination content gaps will appear and make a “squawk” sound. Only when the “squawk” has been identified can you determine the content gap that caused the squawk. From that point it becomes easier to close the “squawk gap” and make that functional process flow more effective and efficient.

    I know the squawk test may sound primordial, but have you thought about how many silos (you know who created) that have blocked the view and allows only the loudest squawks to be heard?

  • Tom Graves March 31, 2010, 11:40 pm

    Re-reading the above and a few of your other posts, seems to me that the core of this is a problem of terminology: ‘business-rules’ versus ‘decision-management’.

    The overall theme is decision-management: what and how and why decisions are made.

    ‘Rules’ describe how decisions are made in the ‘ordered’ regions of decision-making – Cynefin’s Simple and Complicated domains. The key distinction is that rules assume linear cause/effect relationships, and hence that there is a definable ‘correct answer’ and that the same action will always achieve the same result. Most IT-systems and other automation make decisions according to rules, hence ‘business-rules’ is almost synonymous with ‘automatable business’. (Almost, but not quite: rules may be overlaid on rules beyond a point of diminishing-returns, and also some IT-systems are capable of pattern-based rather than rule-based decision-making.)

    ‘Rules’ do not adequately describe how decisions are made in the ‘unordered’ regions of decision-making – Cynefin’s Complex and Chaotic domains, in which cause/effect relationships are either non-linear, are not discernible and/or do not exist (such as in any inherently-unique event), and hence that there is no definable ‘correct answer’ and that the same action may lead to different results. (The inverse is also true here: different actions may lead to the same result.) At present, decision-making in such contexts is generally not amenable to automation, though IT-systems and other automation may be of real value in decision-support.

    So when we’re discussing ‘business rules’, we need to be very clear whether we’re talking about decision-management in general, or the ‘IT-able’ subset which can be used in ‘business-rule engines’ for automated systems. The danger – as is the case with other IT-driven ‘term-hijack’s – is that the subset of ‘business-rules’ may taken as the whole of ‘decision-management’.

    As I mentioned above, decision-management is in some ways a category of knowledge-management, so all the usual KM concerns also apply here – especially the point that the reasoning on which decisions are made can not always be made explicit (which would be a fundamental requirement for automated decision-making).

    Interestingly, one of the key distinctions here rests on teleology – the ‘why’ of decisions. (I was reminded of this this morning when reading the UK Education Department’s guidelines on teaching evolution, creationism and intelligent-design. 🙂 ) In essence, ‘business-rules’ assumes an ordered world in which the question ‘why’ need never be asked; for a parallel example in which business-rules are often applied, note that there is no place in BPMN or UML diagrams for ‘Why’. (Or ‘Where’ – but that’s another story.) There is no teleology to business-rules as such, nor to the internals of how a business-rules engine operates. But in business itself, the question ‘Why?’ matters a very great deal – it’s literally about why we’re there in the first place – hence teleology is central to decision-management.

    Hope this helps, anyway – and thanks again.

  • Paul Vincent April 1, 2010, 11:00 am

    re: note that there is no place in BPMN or UML diagrams for ‘Why’.
    … However there is a lot of interest in the OMG BMM Business Motivation Model to tie such models to goals and motivations. Such motivations also drive decision models, analytic models, business rules etc.

    • Tom Graves April 1, 2010, 12:26 pm

      Paul – as an enterprise-architect, I’d love to see a formal cross-linkage between BMM and BPMN/UML – would be very useful indeed. Is there any sign from OMG that that’s being developed, and if so, when we’re likely to see it?