It's my belief, and I've been writing and speaking about this for quite a while now, that the way we do BI today has reached its limits. Business today demands ever closer to real-time information that must be consistent and meaningfully integrated across ever wider scopes. These demands simply cannot be satisfied by our current concept of a layered, triplicated (and more) data warehouse of hard information--largely numerical data arranged in neat tables--along with some soft information thrown in as an afterthought. The only way forward that I can see is to begin to treat all business information as a conceptually single, integrated, modelled resource with minimal duplication of data. I've described this business information resource (BIR) to a first approximation elsewhere and my seminar will, among other things, dig deeper into the structure of the BIR and the technology needed to create and maintain it.
My current excitement stems from the growing reality of "hybrid" databases--combining the features and strengths of row-oriented and columnar relational databases. Now, I know that academia has proposed approaches to this as much as 8 years ago, but it's only in the last year that commercial databases are introducing it. I wrote about Vertica's FlexStore feature, introduced in 2009, in my last post. The latest announcement I found is of a technology preview program for Ingres VectorWise, the newest entrant in the hybrid database arena. Add Oracle's Exadata V2, announced last year with typical modesty by Larry Ellison as the "fastest machine in the world for data warehousing, but now by far the fastest machine in the world for online transaction processing", and we can see that the approach is finally gaining market traction.
Why is this important? Well, despite the hype, Larry hit the nail on the head. If we finally have databases that can handle both operational and informational workloads equally well, we can begin to define an architecture that doesn't insist on copying vast quantities of data from one database to another. That doesn't mean the death of the data warehouse any time soon, but it does mean that a much more integrated IT environment is coming your way.
Posted March 18, 2010 10:12 AM
Permalink | 2 Comments |




Hi Barry, interesting article. Does this mean that it is your opinion that we now have databases able to handle mixed workloads? Meaning that the database functionality can handle this, rather than throwing in more/better hardware. I would be very interested in your view.
Thank you.
Best regards,
Paul.
Hi Paul,
Good question! My opinion, based on discussions with a few vendors and reading of their white papers is that we may be moving towards databases that are better able to handle mixed workloads. That ability would come from a combination of a better understanding of the mixed workload problem, new software solutions and the application of faster hardware. I'm encouraged by progress on all three fronts, but I don't believe we're there yet, and I expect it will take a year or two before we see fully working products.
Regards,
Barry.