We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

rolls-royce-logo-302.jpgHaving looked at timeliness in Part 1, let's turn our attention to consistency.

Early proponents of data warehousing, including myself, majored on the role of the Enterprise Data Warehouse (EDW) as a repository of a consistent, integrated and historical view of the business.  Leaving aside the historical aspect for now, the desire for consistency and integration can be traced directly to one of the main concerns of decision makers in the 1980s.  There existed a growing proliferation of applications--operational systems--that were responsible for running the business.  These systems were being introduced in an ad hoc manner throughout the business, often on different platforms and addressing different but overlapping aspects of the same process.  

In a bank, for example, a mainframe-based application running against an IMS database handled checking accounts.  A new relational database system running on a minicomputer was introduced to handle savings accounts.  The difficulty for decision makers was to understand the combined account position for individual customers.  The need, stated in a nutshell, was for a "single version of the truth".

This divergence of sources, combined with the often poor data quality in individual operational sources, as well as the need for a single truth, led EDW designers and developers to focus almost maniacally on how to achieve consistency and integration of information in the warehouse.  Enterprise modeling, ETL tools and intricate, often lengthy projects were all used in service of this goal.

Today, we need to pose two important questions.  First, is there really a single version of the truth that can be created and stored in the EDW?  Second, do we have the time and the money to create it?

On the first question, I feel that we have become blinded by our unswerving belief in a universal truth.  Yes, there do exist "truths" in the business that need to be universally agreed.  The quarterly figures announced to the stock markets absolutely need to be internally consistent and well-integrated.  The underlying numbers that lead to these results are similarly constrained.  But, it is equally clear that some numbers can exist as best estimates, close approximations or even "swag" (some wild-assed guess!).  As a culture, we have become obsessed with the second or third decimal point on many numbers.  How many times have you heard election polls being reported with candidates separated by half a percentage point, while the 2% margin of error on the poll is hidden in a footnote?

Answering the first question as we just have leads easily to an answer to the second.  We need to divert resources from seeking complete consistency to achieving consistency where it matters and timeliness where that is important.  And, more, getting the best return on investment in both areas--timelines and consistency.   The real business value in some data lies in its early availability to decision makers; the value in other data resides in its consistency and integrity.

Distinguishing between the two is the key to success.

Join me on my upcoming webinar, "Business Intelligence: the Quicker, the Better", on October 25th for further insights into this important issue.

And for my European readers, allow me to remind you that Larissa Moss is presenting a two-day seminar in Rome on October 20-21st, entitled "Agile Approach to Data Warehousing & Business Intelligence" which will also show how to address this dilemma.


Posted October 18, 2011 8:46 AM
Permalink | No Comments |

Leave a comment