Mike replied to my post about his question on enterprise metadata. He, like me, prefers David Marco’s definition of metadata as “all physical data and knowledge from inside and outside an organization, including information about the physical data, technical and business processes, rules and constraints of the data, and structures of the data used by a corporation” because the business rules pertaining to the data is important. One of the challenges in this broader approach is that the tools that are good for managing each of these different areas are themselves different and they lack integration. While business rule management systems manage rules well and Master Data Management (MDM) solutions are focused on data definitions, the definitions of rules and data are typically semi-detached at best and linked only loosely with things like business processes. Any attempt, therefore, to really manage metadata is going to require multiple products and some flexibility of thinking to use them together. A couple of points:
- Look for repositories with programmable interfaces
These will allow you to link from one to another when you need to, running a query against the MDM repository to explain part of the rules repository, for instance - Think federated not enterprise
I don’t think it is helpful to try and think of all of this information going into some great big repository in the sky. Instead think about focused areas of metadata management and how they might be usefully linked. Even if I need to know something about the data metadata when considering rules, I probably do not need the full depth of information available to someone doing the modeling of that metadata, just a summary of the outcome of that work. - Look for extensible repositories
As these will make it easier to link the various sources together - Start with the end in mind
Don’t try and manage metadata for the whole enterprise or start with the metadata and work out. Instead, start with a specific problem you want to solve and solve that, building enough metadata to do so. For instance, pick a specific set of metrics where inconsistency is a problem and try and solve that by developing a metadata approach for those metrics and then using it.
Mike’s stated goal is to provide both end users (analysts and power users) and developers with access to the logical view of their data – to hide the complexity of the physical representation of the data and expose derived values. That way, whether people are writing code using SAS, Microstrategy, Java, .Net, or if they are using a query tool, they will all get the same answer. And when the business rules of a metric changes, they change it in one place. This is definitely a worthwhile goal but, as I said above, I would not try and solve it across the board but decision area by decision area. Pick an area where inconsistency or complexity is an issue. Work on solving that problem using technologies suitable for widespread adoption. Show the value and get it integrated. Repeat for each successive area. Too much focus on the enterprise-view will detract from visible progress.
My sense is that there are a number of MDM tools (which Mike has begun to use) that would help with this. Mike talks about using MDM tools for initial BPM/SOA projects but not currently for the enterprise. I think this is the right way to start – use the tools to solve very specific problems. A process may not be the right context, a decision may be a better one, but starting with a tight focus is clearly the right approach.
The other point I would make is to think about the role of decision automation in this context. It may be easier to eliminate the complexity by automating a decision than by managing the metadata so that many people can run their own queries. Using a decision service as a way to abstract some of this might work well for certain decisions.
Finally, you might find the work of some folks on Sustainable Architecture (http://sustainableitarchitecture.com) interesting as they are looking at how ETL, MDM and business rules can all be combined to produce a fundamentally sustainable architecture.
I am not sure I really answered his questions but hopefully I helped a little….
