Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

February 2011 Archives

So, another innovative start-up in the data warehousing space has succumbed to the blandishments of a richer, bigger suitor in a Valentine's Day marriage!  Sorry, couldn't resist the obvious parallels as HP and Vertica announced a match made in heaven on February 14th.

Vertica, founded in 2005 by Dr. Michael Stonebraker (Berkley and MIT database guru) and Andrew Palmer, has had an enviable reputation for being a leading innovator and market success in the columnar database field in recent years.  Most recently, they have introduced a hybrid database model on top of the pure columnar database, giving some of the performance advantages of both row-based and column-based models.

On the other hand, HP's status in the data warehousing field is unfortunately most closely tied to the slow-motion train-wreck that was Neoview, whose demise was confirmed in late January.

Of course, we must assume (as presumably HP does) that the Vertica team will bring both their innovative thinking and their undoubted cachet in the specialized data warehouse analytic database market to HP.  The question is: is this a reasonable assumption?

Sadly, in many cases it proves impossible to take external innovation and reputation and successfully embed it into a larger organization.  Whether it is the large organization ethos, the incumbent power structures or the existing technical skills, most acquisitions of smaller technological assets tend to underperform on their buyers' expectations.  And the leadership team of the acquired company, both business and technical, often finds the new environment too challenging, and either leaves as soon as the golden handcuffs are undone or slips namelessly into the larger company culture and forgoes the innovative drive.

While fully understanding the economic realities involved for smaller companies in a market dominated by a relatively small number of enormous players, I am concerned that the trend towards increasing consolidation will kill the innovation we've seen blossoming in the data warehousing space in the past few years.  That would be a great shame, as the business needs that have emerged in recent times demand a significantly different model of business intelligence than we've followed over the past twenty years.  That model, as I've discussed elsewhere, requires advanced innovation from people who understand where we've come from in BI and both the possibilities and limitations of new technologies to solve tomorrow's information challenges.

HP's success with Vertica depends on economic and technological factors, for sure.  However, the most important will undoubtedly be organizational and political in nature.  Will HP, chastened by their clear failure with Neoview, step back and look at the market with fresh eyes and allow Vertica be a change agent in a new and emerging view of business insight?  Or will they attempt to compete with the market incumbents and cast their new Vertica database in the old roles of either buying market presence or playing technical catchup?

In the analytic database marketplace as of now, the mantle of independent innovation must now fall upon ParAccel, Aster Data and a few smaller, mostly open source vendors.

Posted February 15, 2011 3:17 PM
Permalink | No Comments |

Having keynoted, spoken at and attended the inaugural O'Reilly Media Strata Conference in Santa Clara over the past few days, I wanted to share a few observations.

With over 1,200 attendees, the buzz was palpable.  This was one of the most energized data conferences I've attended in at least a decade.  Whether it was the tag line "Making Data Work", the fact it was an O'Reilly event or something else, it was clear that the conference captured the interest of the data community. 

The topics on the agenda were strongly oriented towards data science, "big data" and the softer (aka less structured) types of information.  This led me to expect that I'd be an almost lone voice for traditional data warehousing topics and thoughts.  I was wrong.  While there certainly were lots of experts in data analysis and Hadoop, there was no shortage of both speakers and attendees who did understand many of the principles of cleansing, consistency and control at the heart of data warehousing.

Given the agenda, I was also expecting to be somewhat of the "elder lemon" of the conference.  Unfortunately (in my personal view), in this I was correct.  It looked to me that the median age was well south of thirty, although I've done no data analysis to validate that impression.  Another observation, which was a bit more concerning, was that the gender balance of the audience was about the same as I've seen at data warehouse conferences since the mid-90s: about the same mid-90s percentage of males.  It seems that data remains largely a masculine topic.

The sponsor / vendor exhibitor list was also very interesting.  There were only a few of those that turn up at traditional data warehouse conferences.  Of course, the new "big data" vendors were there in force, as well as a few information providers.  Of the relational database vendors, only ParAccel and AsterData were represented.  Jaspersoft and Pentaho represented the Open Source BI vendors. While Pervasive and Tableau rounded out the vendors I recognized from the BI space.

As a final point, I note that the next Strata Conference has already been announced: 19-21 September in New York.  Wish I could be there!


Posted February 3, 2011 7:02 PM
Permalink | No Comments |