Blog: Dan E. Linstedt

I've wondered about the same question. Fundamentally, they look more like master dimensions to me, rather than a data mart. It harkens back to Kimball and having customer or product dimensions that can conform across the business.

Hi Julius,

I would agree - whether you call them master dimensions or a full master data mart, they are basically for delivery of ALTERED (non-compliant) information.

I would hope that those vendors using the terms do not continue to mix "compliant data warehousing" with MDM. If you see any articles referencing MDM as a compliant data set, I'd love to read through it.

Cheers,
Dan L

Hi Dan,

I think that I agree partially with your views on MDMs, Marts and Warehouses, however, with my backgound, I perhaps have a different perspective.

I treat MDM not only as linked view of the truths of each of the various enterprise systems (ERP, CRM, SCM etc), but also as collection of real-time services to be provided to the enterprise.
Therefore, I agree with the push-pull model you describe, with enterprise systems - but I also see it as a key component of an SOA architecture.
The services I see it providing, are for master record population and retrieval (key for BPMS type activities), Master Object Reference Lookups (key for real-time systems integration scenarios), and providing the source of meta data against which all services should be based (i.e. a method for identifying the single definition of key data entities).

What then becomes more interesting for me, is how this then links to a federated master data ownership, and lifecycle data management approach - to take the management of data as a business asset, away from IT, and back into the hands of the business.

Is this a rant? Possibly!
I would be interested to see if your views on the linkage of these areas is the future too?

Cheers

David J

I don't think there is any question that data warehousing and MDM principles are almost identical - it's just the purpose that might differ. I believe this is supported by the number of data warehouse solution vendors that have made an almost seamless transition to MDM, eg. Kalido and Hyperion.
However, the application of MDM doesn't have to be limited to data warehouse, marts, dashboards and the like. Its extension to transactional applications is appropriate and there is value in achieving it.
I believe the issue is not whether MDM within transactional applications is appropriate, rather the issue is how best to implement across all applications in the enterprise, to achieve the MDM nirvana. I would argue that nirvana is unlikely to be achieved, but there is considerable value in undertaking the journey.

Dan,

It might be helpful to view a "master data mart" as the end result of a "master data management" process. The MDM process accepts whatever data is fed to it from the source systems - the good, the bad, and the ugly. Turn the crank and the data cleansing business rules are applied to produce the MDM mart.

It is possible - as my most recent DW project is doing - to maintain traceability between the "real" (though flawed) data in the source systems and the "clean" (though transformed) data in the MDM mart. Hence the clean data in my DW effectively becomes "gospel" - but the agnostics and skeptics can trace back to the "real" source system as necessary.

I am not comfortable with your implication - if I am understanding your argument - that "gospel" data in your MDM mart isn't a true reflection of the system of record. The "gospel" data in your MDM mart is a better approximation of reality. The business rule transformations in the data cleansing process are a CORRECTION to a previously incomplete or incorrect understanding of a value in the system of record.

I think that the MDM mart, as you describe it, is the BEST system of record - or at least a better one than the "real" system of record. You can incorporate traceability data structures into this MDM mart to tie back to systems of record.

At any rate I believe strongly that MDM - or RDM or CDI or PDM or whatever you want to call this new DW tool - is a very good thing.

Bill

Thank-you all for your comments, they are very thought provoking - I've added MDM Part Deux (II) for those interested in this thread of discussion.

Bill, you raise a question: what is the definition of system of record? I think the lines have blurred and will continue to do so going forward.

I agree that there is value in cleansed and quality merged data. However, I stand by what I say: that a System Of Record "data warehouse" must be an audit trail of raw data (both good and bad) reflected and traceable in the source systems.

Why? because there's a ton of value in understanding what is BROKEN in the business capture and business processes, and not just understanding what good/better/best data looks like from an analytics standpoint. Also, because when the auditor walks in - it's me that ends up in court because I built the system, and the business users I know will point at IT implementation staff and say: "They didn't implement my business rule properly." - ok Maybe I'm jaded just a bit, but I come from a government environment where this actually happened.

Luckily for my team, we implemented a fully compliant and traceable warehouse, and passed the audits with flying colors. We had built "MDM - master data marts" based on the warehouse detail, and the auditor found the business rules to be in-conflict, furthermore they had "data check errors" built in to their operational systems that had been there for 15 years. They found a billing error that amounted to millions of dollars.

This was 12 years ago for me... I've been implementing compliant systems for at least that long, I'm surprised that it took so long for the commercial market to catch up.

After the audit, even the nay-sayers began stating how good the data warehouse was, and how bad the operational systems were at integrating the data. For the first time in 15 years, we saw a RISE in accountability in the business, and a rise in change requests to FIX the business process and business capture problems in the source systems, thus completing the cycle of information quality.

All because we separated the concepts of Master Data Marts and Auditable / Compliant Data Warehousing. I have additional cases like this that I've worked on, one of the secrets to our data warehousing success is the Data Vault data modeling architecture, you can check that out at: http://www.DanLinstedt.com

Great Comments! Thanks,
Dan L

Dan:

I think we are in basic agreement. The MDM without traceability to the "real" source system is something other than the system of record. However, my current project BUILDS this traceability into the MDM - the MDM persists the source values, a source system identifier, time stamp, and the transformed value. Anyone - user or auditor - can easily trace the MDM "clean" data back to its source - within the MDM. Correct me if I am wrong, but I don't think your MDM included this history/traceability in the MDM. I contend that - if this traceability is built into the MDM - then the MDM becomes the system of record. I am not yet sure if I am being pedantic or if my project team has come up with something new and of some significance. Personally I lean towards the second interpretation ... I am often wrong but never in doubt!

Bill

Hi Bill,

Actually I do need to correct you - the "MDM" that we built was in fact compliant, and maintained a full audit trail of every delta of every record across the warehouse - at the lowest level of grain. Again, the government audit would not have been passed had we not had this layering of data over time, in the warehouse "as it stood on the source system". The only thing "master" about our data warehouse were the integrated visions of parts, suppliers, accounts, customers, plans, contracts, employees, and so on. Business keys that made sense at the same semantic level were integrated.

I'd like to chat with you, drop me a personal email and we'll see if we can connect via phone.

Thank-you very much for the kind comments,
Dan Linstedt