Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

November 2012 Archives

IDAA_heart1.jpgWell, perhaps not close to your heart, but certainly close to the heartbeat of your business.  This is a key message of an IBM Virtual Event debuting at 10:30 a.m. EST in the U.S. and 10:30 a.m. GMT / 11:30 a.m. CET in Europe on November 28, where I'll talk about modern mission-critical Business Analytics.

For many businesses, embedding operational analytics in the heart of their OLTP (online transaction processing) applications is a key initiative for 2013. The leaders, of course, have already begun.  The old operational data store (ODS) and operational BI were precursors as far back as the mid-90s, attempting to make faster decisions about operational matters.  These initiatives have had their success stories, but they have been limited by a number of factors, both analytical and operational.  The analytical issue has often been the lack of sufficient quantities of transaction and event data to effective mine.  The operational aspect was the ability to get close enough to the near real-time responses required by business users and customers.  

Both of these issues are being addressed with today's technologies.  The enormous growth of business on the Web in the past decade has meant that customer behavior can be analyzed through clickstreams within websites and linkages across different websites, call centers and more. Such information, analyzed in combination with transaction data, allows retailers to more effectively cross-sell, hotels to increase room occupancy and telcos to reduce churn.  But, for this blog, and the above event, the more interesting point relates to how to close the real-time gap.

Traditionally, business intelligence operates on data that has been extracted from the operational environment and analytic outcomes applied back to that environment afterwards. In short, the data is brought to the analytics.  This approach introduces significant delays.  An obvious solution would be to bring the analytics to the data; however, prior technology did not easily allow that.  I discuss this in terms of the mainframe, System z, environment, but the principle applies elsewhere too.

It is an oft-forgotten fact that 70% of all data transactions in the banking, insurance, retail, telecommunications, utilities and government industries still occur on the System z platform, due to its performance, cost, reliability and security characteristics.  The inclusion of the Netezza-powered IBM DB2 Analytic Appliance within the System z complex creates a system with a dual personality -transactional performance of the original environment combined with the analytic performance of Netezza required for integrated operational analytics.  With the inclusion of SPSS Predictive Analytics on Linux and Cognos on the zOS and Linux platforms, the need to move data out of the System z environment is largely eliminated.  More details are to be had in the Virtual Event where IBM's Dan Wardman and David Jeffries will fill in the technical details. See also my White Paper, "Integrating Analytics into the Operational Fabric of Your Business, A combined platform for optimizing analytics and operations".

Irrespective of platform, it is becoming increasingly clear that when it comes to operational decisions, they have to come from the heart rather than the head!



Posted November 27, 2012 12:54 AM
Permalink | No Comments |
Big Data RIP tombstone.jpg2012 begins to wind down.  Yes, I know it's still only mid-November, but I find it hard to avoid thinking of year-end when the retail industry has been pushing Christmas for weeks already.  I've been preparing for my keynote at Big Data Deutschland in Frankfurt (20-21 Nov) next week, so it seemed appropriate to share some thinking on where big data is at now.  Also, I've been deeply involved in analyzing the results of the EMA / 9sight big data survey which has just been published.  My bottom line?  Big data is dead!

Of course, I don't mean that literally.  What I'm really trying to do is to get the attention of the marketing folks who have been using and abusing the term, particularly during 2012.  Two very clear results emerge from the big data survey when it comes to real customer projects carrying the moniker big data.  

First, the industry has been besotted by size.  Carefully avoiding now all vaguely salacious phrases, the fact is that size is so relative that calling data big or small is more about bragging or shaming than any measure of real use.  Our survey showed that 60% of respondents were managing less than 100TB of data in total in their organizations, while only 5% stretched beyond a petabyte.   Not all of this data was part of their big data projects; on average, only some 30% was included there.  This strongly suggests that so called big data technology is being widely used for something other than processing excessively large data volumes.

Second, it's not all about exotic types of data either.  Yes, some 45% of the data sources fall under the category of human-sourced information, which includes social media sources.  But, just over 30% is process-mediated data -- transactional data gathered and created in traditional operational and informational applications.  For a more detailed explanation of these data domains, as I call them, please see my recent White Paper "The Big Data Zoo - Taming the Beasts, The need for an integrated platform for enterprise information".  So, big data projects are addressing a substantial proportion of the data we've known and loved for many years.

You can hear more of the survey results on the EMA / 9sight webinar on Thursday, 13 December, 11 a.m. PST / 2 p.m. EST.

What is actually becoming important as we look towards 2013 is what businesses are really doing with data at the moment that is different from what they've traditionally done.  I believe there are two distinct trends.  One is, of course, business analytics.  This is simply an evolution of traditional BI, with more of an emphasis on exploration (or mining) and less on reporting and dashboards.  The second is more interesting and, potentially, game changing.  This involves the re-integration of operational action taking and informational decision making in customer-facing applications that automatically modify their behavior in real-time in response to rapidly changing market or personal circumstances.

All this says to me that big data as a technological category is becoming an increasingly meaningless name.  Big data is essentially all data.  Is there any chance that the marketing folks can hear me?


Posted November 13, 2012 11:11 AM
Permalink | No Comments |