Ah â€“ the much maligned and misunderstood Operational Data Store (ODS)â€¦ I am at the Data Warehousing Institute conference in San Diego this week and I heard an astounding thing â€“ a declaration that the ODS was dead. Gee, I must have missed the obitâ€¦
Why was the ODS declared dead? Because many people mistakenly believe that the ODS is a staging area for the data warehouse. Whether or not you need a staging area for the raw data coming into a data warehouse is another discussion altogether but letâ€™s stick with the ODS for this blog.
OK â€“ the basics. What is an ODS? In the mid-1990â€™s, the ODS was introduced by Bill Inmon and myself in our book, â€śBuilding the Operational Data Storeâ€ť. In it, we defined the ODS as a â€śsubject-oriented, integrated, current, volatile collection of data used to support the tactical decision-making process for the enterprise.â€ť
Letâ€™s go through it slowly to make sure everyone gets the definition:
1. Subject-oriented â€“ The ODS is built similarly to the data warehouse in that its data content is focused about different data subjects. Examples of data subjects are customer, product, and order data.
2. Integrated â€“ the data from each subject area is fully integrated â€“ again much like the data in the data warehouse. It is cleaned up as much as is technologically and humanly possible during the ETL process before it is populated into the store.
3. Current â€“ the data in the ODS is as current as we can technologically make it â€“ a significant difference from a traditional data warehouse. Current as in as close to â€śreal timeâ€ť as we can get. The data probably wonâ€™t be synchronous with the operational system source because of the latencies inherent with integration but it will be close.
4. Volatile â€“ major differentiator from a data warehouse. The new or changed data flowing into the ODS updates the appropriate records or fields exactly like you would update a record in an OLTP operational system. For example, when a new customer address is brought in, the old customer address is overwritten. An audit trail is created to trace the change but the old data is gone. If you want the history of the customerâ€™s moves, you will have to either go to the audit trail or the data warehouse where history is preserved in snapshots.
It is this last aspect of the ODS that differentiates it significantly from a data warehouse and puts it in the camp of operational systems. Because of this feature, referential integrity must be fully implemented. Cascading updates and deletes, complete edit checks, and so on are needed just as they are in operational systems.
I just donâ€™t get it. Does this sound like a holding area for extracts from operational systems? Donâ€™t think soâ€¦ In fact, it sounds an awful lot like the newly invented customer hub (CDI fame) if that hub contains only current customer data. However, the ODS was originally developed to integrate both current master data and current transactional data â€“ the latter inclusion making it broader in purpose than the CDI hub. In any case, it was NEVER defined as a staging area for the data warehouse. I suggest that those calling their staging areas an ODS start calling those components by their rightful name. That would certainly help clear up the confusion.
The ODSâ€™ purpose ranges from producing operational reports to propagating operational data for downstream operational systems to supporting the migration of legacy systems. And, of course, like any other operational database â€“ it can be a source of data for the data warehouse. Perhaps it was ahead of its time when we first introduced it but to malign such a useful component in your arsenal of integration capabilities is just a shame.
Hopefully this short tutorial on what an ODS is has helped dispel the incorrect notions floating around. I feel like Mark Twain who said â€śThe reports of my death are greatly exaggeratedâ€¦â€ť The ODS is not dead â€“ it is alive and kicking more than ever helping companies integrate their operational data for operational BI and master data management.
If you are interested in a white paper I wrote on the ODS (lots more detail about what it is, how to build one, its uses, and how to get going), just send an email at CImhoff@IntelSols.com. I'll be happy to send it to you.
Long live the ODS!
Yours in BI (and ODS) success,