We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

Big Data RIP tombstone.jpg2012 begins to wind down.  Yes, I know it's still only mid-November, but I find it hard to avoid thinking of year-end when the retail industry has been pushing Christmas for weeks already.  I've been preparing for my keynote at Big Data Deutschland in Frankfurt (20-21 Nov) next week, so it seemed appropriate to share some thinking on where big data is at now.  Also, I've been deeply involved in analyzing the results of the EMA / 9sight big data survey which has just been published.  My bottom line?  Big data is dead!

Of course, I don't mean that literally.  What I'm really trying to do is to get the attention of the marketing folks who have been using and abusing the term, particularly during 2012.  Two very clear results emerge from the big data survey when it comes to real customer projects carrying the moniker big data.  

First, the industry has been besotted by size.  Carefully avoiding now all vaguely salacious phrases, the fact is that size is so relative that calling data big or small is more about bragging or shaming than any measure of real use.  Our survey showed that 60% of respondents were managing less than 100TB of data in total in their organizations, while only 5% stretched beyond a petabyte.   Not all of this data was part of their big data projects; on average, only some 30% was included there.  This strongly suggests that so called big data technology is being widely used for something other than processing excessively large data volumes.

Second, it's not all about exotic types of data either.  Yes, some 45% of the data sources fall under the category of human-sourced information, which includes social media sources.  But, just over 30% is process-mediated data -- transactional data gathered and created in traditional operational and informational applications.  For a more detailed explanation of these data domains, as I call them, please see my recent White Paper "The Big Data Zoo - Taming the Beasts, The need for an integrated platform for enterprise information".  So, big data projects are addressing a substantial proportion of the data we've known and loved for many years.

You can hear more of the survey results on the EMA / 9sight webinar on Thursday, 13 December, 11 a.m. PST / 2 p.m. EST.

What is actually becoming important as we look towards 2013 is what businesses are really doing with data at the moment that is different from what they've traditionally done.  I believe there are two distinct trends.  One is, of course, business analytics.  This is simply an evolution of traditional BI, with more of an emphasis on exploration (or mining) and less on reporting and dashboards.  The second is more interesting and, potentially, game changing.  This involves the re-integration of operational action taking and informational decision making in customer-facing applications that automatically modify their behavior in real-time in response to rapidly changing market or personal circumstances.

All this says to me that big data as a technological category is becoming an increasingly meaningless name.  Big data is essentially all data.  Is there any chance that the marketing folks can hear me?

Posted November 13, 2012 11:11 AM
Permalink | No Comments |

Leave a comment


Search this blog
Categories ›
Archives ›
Recent Entries ›