Blog: Barry Devlin Subscribe to this blog's RSS feed!

Barry Devlin

As one of the founders of data warehousing back in the mid-1980s, a question I increasingly ask myself over 25 years later is: Are our prior architectural and design decisions still relevant in the light of today's business needs and technological advances? I'll pose this and related questions in this blog as I see industry announcements and changes in way businesses make decisions. I'd love to hear your answers and, indeed, questions in the same vein.

About the author >

Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

August 2008 Archives

I was browsing through the blogs on B-eye-network.com this morning (Sunday - yeah, sad, I know) and came across two recent entries that spoiled my coffee. Given that I'm no fan of instant gratification (in IT anyway), I'm not going to give you links, so you have to work at finding them yourself. But the phrases that caught my eye were "Instant SOA", "Data marts in about an Hour" and "full EDW's with AS-IS star schemas in 2 weeks".

Now I'm as fond of a shortcut as the next guy, but I've learned the the word "Instant" is not all goodness. When I've bought some instant Spaghetti Bolognese in the local supermarket I've found that the cost is a lot higher than the individual ingredients and the taste, well, leaves a lot to be desired. Sure, I saved some time when I got home, but did I get value for money? And did I end up with what I really wanted? So, why should I expect more from an Instant DW?

"Caveat emptor" as the Romans used to say. Here are a few contra-indications for when instant gratification should not be expected in your next BI (or SOA) project:

  1. The business users are not quite sure what they want.
    Most BI projects start with a vague set of requirements from the potential users. It's going to take some time to hone these down to a usable definition of data and query needs. In the meantime, maybe it's best to let the users continue to play with their instant Excel spreadsheets and look over their shoulders to see what they're doing.
  2. Somebody forgot to document the meanings of the data in the source applications.
    This is the oldest metadata problem. If your data sources have not been properly described, an Instant DW is likely to be instantly dismissed as misleading and inaccurate. Do you want to go there?
  3. Garbage in, garbage out. Or worse...
    If your ingredients (data sources) are contaminated with erroneous data, you're going to end up with a very sick business on your hands if you just take the Instant DW approach. Understanding and fixing dirty data is time-consuming, but mandatory.

It's all about quality time... or quality vs. time. If I bring home my instant Spaghetti Bolognese, I may get it on the table within a few minutes. But, if the kids won't eat it or, worse, throw up that night, I'd argue I've made the wrong trade-off between time and quality. You need to consider the same balance in a BI or SOA project.

Now, I'm off to spend some quality time with my kids :-)


Posted August 24, 2008 12:20 PM
Permalink | No Comments |

I was at the Business Object Summit this week in Boston, where the main emphasis was on linking strategy to execution and a seeming focus on the larger enterprises. All very SAP-inspired, I thought. And very insightful, especially if you're a large enterprise. There have been some comments in the blogs already on these topics. But it was a small conversation over lunch that caught my interest...

Information OnDemand. No, not the annual IBM Conference in Las Vegas, in October. But a rather low key effort from Business Objects with a website to allow companies to access market data and incorporate it into their BI efforts.

There's a definite growing interest these days in combining external data with the contents of the warehouse. But it does raise some concerns, not least about the reliability of the external data and how to create a valid semantic relationship between the two data sets. In the past, companies have addressed these concerns by obtaining key market and other external data from trusted sources like Dunn and Bradstreet, Reuters and others and then ensuring that such data entered the warehouse via a controlled feed designed by Information Architects who could match the two data sets correctly. After all, such external data is another information source for the warehouse and should be managed like any other.

This method works well for large enterprises with a centrally-controlled approach to the warehouse. And where the value-add derived from or risks incurred by using this data are significant, this method is probably still required. But what if you are a small or medium enterprise? Or what if you really only want to do a couple of once off analyses?

Shopping at the Information OnDemand website appears to be the answer! Here you can buy prebuilt, but customizable, reports combining your data with external market and financial data. You can buy one-time snapshots or subscribe for regular updates.

For larger companies, this could provide a safe and cost-effective way of dipping their toe in the big ocean of external data. For smaller companies, it could be all they need. Sounds like a useful idea to me!

The service has been available since September 2007, but I hadn't come across it before. Maybe there are some similar services I should know about, so please feel free to comment.


Posted August 14, 2008 5:14 PM
Permalink | No Comments |

I came across an ad today for a Google Webcast on Universal Search for Business. It contained the phrase "As the volume of information inside enterprises explodes, most executives recognize the importance of a Google-like search solution for business content.", which set me wondering...

A Google-like search solution for business content? What exactly does that mean?

The phrase "Google-like search", of course, covers a multitude of marketing-speak, but let's assume that it includes the patented PageRank technology behind Google's Internet search success. Google itself describes PageRank as follows: "PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value." (http://en.wikipedia.org/w/index.php?title=PageRank&oldid=230400158 as of Aug. 7, 2008). A number of questions arise for me: Does an enterprise intranet usually have a vast link structure? Would business executives really consider the "democratic vote" of the organization as a valid indicator of a document's importance? Indeed, how democratic is the link structure in an intranet?

Google, Wikipedia and many Web 2.0 systems have an underlying belief in James Surowiecki's concept of "the wisdom of crowds". Data warehousing, Business Intelligence and, indeed, all traditional IT development tend to put more faith in experts and their accumulated knowledge. In the BI world, I'm beginning to see some level of acceptance that the so-called experts do not have a monopoly on business knowledge. We see that there is a growing need to allow and, indeed, facilitate the feedback of knowledge that emerges on the fringes of the BI community (the front-line staff and first-line managers) back into the core of the warehouse for wider promulgation and reuse.

But, to what extent does Google and the Web 2.0 community recognize that some knowledge is inherently more useful or valuable (although not necessarily "right") simply based on the authority of its source? And within the tighter and more closed confines of an enterprise, that not all the requirements for wise crowds are met? If not, we may see the many years of careful effort by data modelers and administrators, and information stewards overturned in the rush to Web 2.0. This would not be in anybody's interest.

On the other hand, if I've made the wrong assumption about what "Google-like search" means... Anybody care to comment? Or maybe I'll find time to sign up for the webinar!


Posted August 7, 2008 8:17 PM
Permalink | No Comments |