We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

October 2013 Archives

The seventh in a series of posts introducing the concepts and messages of my new book, "Business unIntelligence--Insight and Innovation Beyond Analytics and Big Data", now available!

Fig 9-4 - Rodin The Thinker.jpgIt has long been assumed in the BI community that more information is "a good thing" when it comes to making better decisions.  Except when there is too much information...when we encounter information overload. Some qualify the thinking by requiring information to be relevant, although that raises questions of how to determine relevance, especially in entirely novel or poorly understood situations. Despite any such reservations, the BI industry generally focuses entirely on information-related issues, from preparation to presentation, and leaves any thinking about the process of how humans make decisions to some unidentified other party.

So, how does insight arise in the human process of decision making? Of course, information does play an important role, but the information we focus on in BI is but a very thin sliver of the actual information that the mind takes into account. Today, neurobiology and psychology tells us that our minds are absorbing information from the earliest days of life, and that such pre-verbal information has significant and unconscious impact on all of the decisions we make during our lives. Such early information conditions our social behavior and expectations of the world. In a similar, although probably in a somewhat more conscious manner, later life experiences create an informational background that colors our thinking in each and every decision we take.

BI purists may argue that such information is unquantifiable and therefore should be discounted.  However, its impact can be large and may be inferred from the ongoing behavior of people involved in the decision making process. When we work in teams, we automatically notice this information and adjust accordingly. Joe always brings up such-and-such a problem. Maria regularly took issue with the previous CFO, but supports the current one, even though the policies and practices are unchanged. This informal information could be gathered electronically and mined for patterns even today to create an entirely new view of how irrational most decision really are.

This leads us to look beyond information as the sole or, even, majority basis for decision making. Rational choice theory has long held sway as the foundation of thinking about business decision making. In recent years, the roles of intuition, gut-feeling, emotional state and intention are slowly coming to the fore as possible contributors.  The BI community has yet to catch up on such thinking. And, indeed, its integration into decision support (to recall that old phrase) requires software and methods that are far from those found in traditional and current BI.

Collaborative and social tools come closest to the foundation of what is required, and I'll address this as we look at innovation in the next article.

Thumbnail image for Thumbnail image for Business unIntelligence Cover.jpg
I will be further exploring the themes and messages of "Business unIntelligence: Insight and Innovation Beyond Analytics and Big Data" over the coming weeks.  You can order the book at the link above; it is now available. Also, check out my presentation at the BrightTALK Business Intelligence and Big Data Analytics Summit, recorded Sept. 11 and "Beyond BI is... Business unIntelligence" recorded Sept. 26. Read my interview with Lindy Ryan in Rediscovering BI.

Upcoming conference appearances where I can share more are in Los Angeles at SPARK! on Oct. 15-16 and in Rome at a dedicated two-day seminar on October 30-31.

Posted October 11, 2013 11:36 AM
Permalink | No Comments |
The seventh in a series of posts introducing the concepts and messages of my new book, "Business unIntelligence--Insight and Innovation Beyond Analytics and Big Data", now available!

Fig 9-4 - Rodin The Thinker.jpgIt has long been assumed in the BI community that more information is "a good thing" when it comes to making better decisions.  Except when there is too much information...when we encounter information overload. Some qualify the thinking by requiring information to be relevant, although that raises questions of how to determine relevance, especially in entirely novel or poorly understood situations. Despite any such reservations, the BI industry generally focuses entirely on information-related issues, from preparation to presentation, and leaves any thinking about the process of how humans make decisions to some unidentified other party.

So, how does insight arise in the human process of decision making? Of course, information does play an important role, but the information we focus on in BI is but a very thin sliver of the actual information that the mind takes into account. Today, neurobiology and psychology tells us that our minds are absorbing information from the earliest days of life, and that such pre-verbal information has significant and unconscious impact on all of the decisions we make during our lives. Such early information conditions our social behavior and expectations of the world. In a similar, although probably in a somewhat more conscious manner, later life experiences create an informational background that colors our thinking in each and every decision we take.

BI purists may argue that such information is unquantifiable and therefore should be discounted.  However, its impact can be large and may be inferred from the ongoing behavior of people involved in the decision making process. When we work in teams, we automatically notice this information and adjust accordingly. Joe always brings up such-and-such a problem. Maria regularly took issue with the previous CFO, but supports the current one, even though the policies and practices are unchanged. This informal information could be gathered electronically and mined for patterns even today to create an entirely new view of how irrational most decision really are.

This leads us to look beyond information as the sole or, even, majority basis for decision making. Rational choice theory has long held sway as the foundation of thinking about business decision making. In recent years, the roles of intuition, gut-feeling, emotional state and intention are slowly coming to the fore as possible contributors.  The BI community has yet to catch up on such thinking. And, indeed, its integration into decision support (to recall that old phrase) requires software and methods that are far from those found in traditional and current BI.

Collaborative and social tools come closest to the foundation of what is required, and I'll address this as we look at innovation in the next article.

Thumbnail image for Thumbnail image for Business unIntelligence Cover.jpg
I will be further exploring the themes and messages of "Business unIntelligence: Insight and Innovation Beyond Analytics and Big Data" over the coming weeks.  You can order the book at the link above; it is now available. Also, check out my presentation at the BrightTALK Business Intelligence and Big Data Analytics Summit, recorded Sept. 11 and "Beyond BI is... Business unIntelligence" recorded Sept. 26. Read my interview with Lindy Ryan in Rediscovering BI.

Upcoming conference appearances where I can share more are in Los Angeles at SPARK! on Oct. 15-16 and in Rome at a dedicated two-day seminar on October 30-31.

Posted October 11, 2013 11:36 AM
Permalink | No Comments |
The sixth in a series of posts introducing the concepts and messages of my new book, "Business unIntelligence--Insight and Innovation Beyond Analytics and Big Data", now available!

Pillars.jpgIn my last post, I introduced the three pillars of the REAL logical architecture of Business unIntelligence. To review briefly, the central, process-oriented data pillar contains traditional operational and core informational data, fed from legally binding transactions (both OLTP and contractual). It is centrally placed because it contains core business information, including traditional operational data and informational EDW and data marts in a single logical component. Machine-generated data and human-sourced information are placed as pillars on either side. The leftmost pillar focuses on real-time and well-structured data, while the one on the right emphasizes the less-structured and, at times, less timely information.

The concept of the pillars emerged from my struggle to clarify the concept of big data and how it relates to the data that businesses have been collecting since we began automating the processes of first running and later managing the business via computers some 50 or more years ago. The pillars are, to a first approximation, delineated my processing and storage concerns. However, there is a more fundamental but less obvious distinction that relates to the sources of the data and information stored and used. These sources are represented by the four clipped boxes at the bottom of the picture.

Let's start in the middle with transactions. Although we often think of transactions in a technical context, more fundamentally they stand for the legally binding agreements (or valid steps towards such agreements) on which all business is based. When we recognize this, we see that traditional operational and informational systems are designed to collect, manage and monitor the contractual and formal information of the business. Before the emergence of big data, we seldom considered in any detail what happened before such transactions occurred.

This thinking leads to the three fundamental information/data sources at the bottom of the picture. Measures and events come from the physical world of machines and show ongoing conditions (e.g. temperature, velocity, location) and changes in conditions (e.g. an acceleration, a button pressed, a call ended).  Messages are human-sourced communications in text, voice, image or video that represent something that one person wants to share with another.  All of these can and do generate transactions when processed in traditional system. Big data simply records them and analyzes them as "ambient data", as Dale Roberts calls it in Decision Sourcing. Because it precedes transactions and, in many cases, does not lead to any transactions, such information leads to a very different view of the world than our traditional systems provide. Where it precedes transactions, we can do predictive analytics about how the formal business will be affected. When it is completely unrelated to transactions, we can gain new insights into reality that can drive true innovation.

I'll talk more about insight and innovation in my next post...

Thumbnail image for Thumbnail image for Business unIntelligence Cover.jpgI will be further exploring the themes and messages of "Business unIntelligence: Insight and Innovation Beyond Analytics and Big Data" over the coming weeks.  You can order the book at the link above; it is now available. Also, check out my presentation at the BrightTALK Business Intelligence and Big Data Analytics Summit, recorded Sept. 11 and "Beyond BI is... Business unIntelligence" recorded Sept. 26.

Upcoming conference appearances where I can share more are in Amsterdam at the Business Analytics Congress on Oct. 9 and in Los Angeles at SPARK! on Oct. 15-16.

Posted October 4, 2013 3:03 AM
Permalink | No Comments |