Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

I'm presenting a two-day seminar for Technology Transfer in Rome in mid-April, entitled "BI2--From Business Intelligence to Enterprise IT Integration" and am currently researching and preparing the material.  And the more I research, the more excited I get about the prospects for the next wave of development from BI to... what?  Well, that's the real question for me!

It's my belief, and I've been writing and speaking about this for quite a while now, that the way we do BI today has reached its limits.  Business today demands ever closer to real-time information that must be consistent and meaningfully integrated across ever wider scopes.  These demands simply cannot be satisfied by our current concept of a layered, triplicated (and more) data warehouse of hard information--largely numerical data arranged in neat tables--along with some soft information thrown in as an afterthought.  The only way forward that I can see is to begin to treat all business information as a conceptually single, integrated, modelled resource with minimal duplication of data.  I've described this business information resource (BIR) to a first approximation elsewhere and my seminar will, among other things, dig deeper into the structure of the BIR and the technology needed to create and maintain it.

My current excitement stems from the growing reality of "hybrid" databases--combining the features and strengths of row-oriented and columnar relational databases.  Now, I know that academia has proposed approaches to this as much as 8 years ago, but it's only in the last year that commercial databases are introducing it.  I wrote about Vertica's FlexStore feature, introduced in 2009, in my last post.  The latest announcement I found  is of a technology preview program for Ingres VectorWise, the newest entrant in the hybrid database arena.  Add Oracle's Exadata V2, announced last year with typical modesty by Larry Ellison as the "fastest machine in the world for data warehousing, but now by far the fastest machine in the world for online transaction processing", and we can see that the approach is finally gaining market traction.

Why is this important?  Well, despite the hype, Larry hit the nail on the head.  If we finally have databases that can handle both operational and informational workloads equally well, we can begin to define an architecture that doesn't insist on copying vast quantities of data from one database to another.  That doesn't mean the death of the data warehouse any time soon, but it does mean that a much more integrated IT environment is coming your way.

Posted March 18, 2010 10:12 AM
Permalink | 2 Comments |

2 Comments

Hi Barry, interesting article. Does this mean that it is your opinion that we now have databases able to handle mixed workloads? Meaning that the database functionality can handle this, rather than throwing in more/better hardware. I would be very interested in your view.

Thank you.

Best regards,

Paul.

Hi Paul,
Good question! My opinion, based on discussions with a few vendors and reading of their white papers is that we may be moving towards databases that are better able to handle mixed workloads. That ability would come from a combination of a better understanding of the mixed workload problem, new software solutions and the application of faster hardware. I'm encouraged by progress on all three fronts, but I don't believe we're there yet, and I expect it will take a year or two before we see fully working products.
Regards,
Barry.

Leave a comment