Blog: Dan E. Linstedt Subscribe to this blog's RSS feed!

Dan Linstedt

Bill Inmon has given me this wonderful opportunity to blog on his behalf. I like to cover everything from DW2.0 to integration to data modeling, including ETL/ELT, SOA, Master Data Management, Unstructured Data, DW and BI. Currently I am working on ways to create dynamic data warehouses, push-button architectures, and automated generation of common data models. You can find me at Denver University where I participate on an academic advisory board for Masters Students in I.T. I can't wait to hear from you in the comments of my blog entries. Thank-you, and all the best; Dan Linstedt http://www.COBICC.com, danL@danLinstedt.com

About the author >

Cofounder of Genesee Academy, RapidACE, and BetterDataModel.com, Daniel Linstedt is an internationally known expert in data warehousing, business intelligence, analytics, very large data warehousing (VLDW), OLTP and performance and tuning. He has been the lead technical architect on enterprise-wide data warehouse projects and refinements for many Fortune 500 companies. Linstedt is an instructor of The Data Warehousing Institute and a featured speaker at industry events. He is a Certified DW2.0 Architect. He has worked with companies including: IBM, Informatica, Ipedo, X-Aware, Netezza, Microsoft, Oracle, Silver Creek Systems, and Teradata.  He is trained in SEI / CMMi Level 5, and is the inventor of The Matrix Methodology, and the Data Vault Data modeling architecture. He has built expert training courses, and trained hundreds of industry professionals, and is the voice of Bill Inmons' Blog on http://www.b-eye-network.com/blogs/linstedt/.

My last post generated some great responses, ranging from Master Data Management as MDM philosophy to MDM Data Marts as "better than gospel" or better than system of record on the source system. In this entry I will take a look at MDM in a little more detail, and try to answer my view points on some of the issues raised in the responses. First, thank-you to all those who are responding, I enjoy reading your thoughts.

There is an interesting twist to compliance in one of the responses I received, the fact that a Master Data Mart (as I would call it) is better than the System of Record data, and should be taken (and often is) as Gospel to the business. Let's delve into the pros and cons of calling a Master Data Mart (altered, integrated, cleansed data) Gospel to the business. These by no means are complete lists, just a few thoughts running through my mind.

Pros:
1. Better information quality (in some cases) - but who's to say the Quality engine that is applying the rules, or the ETL/T rules are "better" than the data set that was actually entered? If a business changes data that I enter without checking with me first, I consider that an offensive move - why? Because the next time I contact the business they insist that their information is right, and I am wrong. Talk about loss of customer loyalty! This is just one of things we have to watch out for. Who's judging Better or Worse?
2. Single view of the enterprise data set - can't argue with that, there definitely is value to having single consolidated lists of data that all say the same thing... What we have to watch out for here is the security and governance of the data set - who's using it, when, where, and how are they applying it? Is it a good thing? From an IT perspective it's a great thing, when used properly in business it's also a great thing.
3. Lower cost to manage - with consolidation, comes a single store of information centered around business - so it is possible to have a lower overall cost of management of the systems, the data stores, and the routines that deliver the data.
4. Ease of implementation of EII, and SOA - with consolidation comes a necessity to implement metadata, and metadata management, and with that comes a level of understanding that assists in the build out of consistent EII and SOA systems.

Cons:
1. Too much of a "Gospel" thing - watch out! Altered data is NOT necessarily always good, I've got some examples of the insurance industry taking my information, interpreting it - they way they think it should be, then abusing it, in marketing, as well as claim cases. Besides that, someone who knows the consolidation routines can alter the data sets in a way like you've never seen. Most of us though are not bad apples by default, so this would never happen right?
2. Beware of the Business User shirking the system - if there is a way, there will be individuals who will find the loop-holes. With certain consolidations, loss of detail occurs - with a loss of detail and a merge, there are bound to be loop-holes in the system, ways in which certain individuals can learn the patterns, and overcome the "edit checks" or consolidation routines. But of course, we have error marts, and metrics to monitor these problems right?
3. Gospel doesn't mean "Audtable" - just because the business thinks the data is better, doesn't necessarily mean it's auditable or can be called a warehouse. The consolidated data often looses detail information, and traceability. In doing so, the auditors can (and have) take the company to court. Remember that implementation experts in IT are the ones who built the routines, and the business is never wrong... I've seen it happen first hand.
4. it’s just as important to know what's BROKEN as it is to gain a consistent view of the business information. I don't object to Master Data Marts, in fact, I think they're a great thing. I object to the loss of detail - at a data warehousing level. I've saved a specific company over $45 million USD in 6 months through a stringent warehouse process, yes we produced MDM over 10 years ago, but it was for data marts. By the way, we also produced a Master Error List (error mart) in which the business (for the first time in 15 years) began FIXING the broken business processes in the source systems. Why? Because they finally began to have visibility into what it was costing them to have "broken" operational systems - all because we tracked errors on a detailed and auditable level.

I believe that MDM and Master Data Marts are a wonderful thing, they do have value in business - and I highly recommend undertaking these efforts. However I still do NOT believe they should be a part of an Auditable Data Warehouse where detailed level data should exist and remain traceable. Also let's take a look at Master Data Management, and what it means....

MDM has the phrase "Data Management" embedded within. IBM just announced $41 Billion investment to be in the Data Management Space (or some huge $$ figure like that). Data Management is absolutely grand, wide vision - ultra-wide and multi-spectrum. Before you can "master" data management, you must master the data management side of the house. Maybe the acronym isn't so good after-all? In another post we'll explore just what Data Management means (at least from the Web's point of view). I'll also come back to physically federated data marts (MDM style), versus consolidated Master Data Marts (in a single physical location), and discuss the pros and cons of these aspects.

Feel free to weigh in with your pros' and con's - if we can build a substantial list of pro's and con's for MDM as Gospel, I'll write an article dedicated to it, and list all the sources. I think it would be good for everyone to have a say in this.

Thanks, see you next time,
Dan L


Posted February 28, 2006 5:35 AM
Permalink | 2 Comments |

2 Comments

Dan:

Actually IBM's investment in Information on Demand (IoD) is only $1 billion. I am hoping that the remaining $40B that you identified appears in my bonus check ;-)) IBM's Master Data Management service is evolving but appears to be considerably more ambitious than the functionally more narrow DW MDM that you have been discussing in your column. I am not suggsting this is a good or bad thing - only that the reader should be prepared for a different interpretation of MDM from IBM.

In a recent IBM article (see URL below) "In IBM's view, MDM is a set of disciplines, technologies, and solutions used to create and maintain consistent, complete, contextual, and accurate business data for all stakeholders (users, applications, data warehouses, processes, enterprises, trading partners, and so on). It's a holistic framework for managing structured and unstructured data that's aligned with business processes and managed throughout the information life cycle ... " IBM's MDM is built with SOA in mind. Users request a service (Add customer, record product sale, etc) from this MDM.

Bill

http://www-306.ibm.com/software/info/middleware/masterdata/index.jsp

Hi Bill,

Thank-you so much for the excellent feedback. I'm sorry I didn't get the investment $$ figure right. None-the-less, $1B is still a significant amount of money, no matter what the initiative.

Regarding the DW MDM I'm discussing, Data Management is but one piece. The light finally dawned on me the other day, when I sat pondering "what does MDM really mean?" And of course, the answer hit me smack dab in the face - it means "data management", and could be potentially on a grand scale of "a journey" not a destination, not a single table, but a manner in which business is done. It can include both accountable data warehouses and data marts, producing more than "one" copy of the "master list", of course, only one (and the lowest level of grain at that) copy can actually be deemed the "master" if we want to put it that way.

I'd love to hear about some of the issues (both business and technical) that you are encountering with your MDM efforts.

Thanks,
Dan L

Leave a comment

    
Search this blog
Categories ›
Archives ›
Recent Entries ›