BeyeNETWORK - The Global Vision for BI and Beyond

A Long Time Ago

Success Factors for Data Warehousing

Published: October 2, 2008

Bill Inmon discusses the reasons that some organizations are more successful with software, focusing on data warehousing success.

A long time ago, I had one of the best jobs in my life. I worked for Amdahl Corporation, and I was the leader of group of people who did what we called design review. With a few other Amdahl employees, I went around the world (literally) and spent a week with Amdahl customers and prospects. We looked at their online systems and made suggestions to improve performance and availability. We looked at IMS, CICS, VSAM, IDMS and other types of systems. While we always started with performance and availability, these design reviews often led into other interesting areas. After a week, we took the consolidated notes and wrote a letter to the client with our suggestions. Collectively, we went all over the U.S., Mexico, Europe, New Zealand and Australia. It was my real “education.”

One of the observations that we made was that the success with the software was not a function of the software. Some organizations were very successful with IMS, for example, where other organizations just could not get IMS to do what was needed. After seeing this phenomenon repeatedly, we were led to the conclusion that success had very little to do with the technology that was being used. Success had everything to do with the organization, its leadership, and its skill.

In fact, if we had to boil down the factors that dictated success, they would have been –

• An understanding of architecture
  
  
• Discipline
  
  
• Design

But the biggest factor separating successful shops from unsuccessful shops was the desire and ability to look ahead and understand the future. Successful shops had a good grasp of what the future of their organization and the deployment of technology in their organization looked like while unsuccessful shops had no understanding of the future. Unsuccessful shops simply operated on a day-to-day basis being driven by events in a reactive manner. Successful shops were proactive.

This lesson was observed over a period of about three years in company after company. In the trade press, people complained about the technology. But in practice, some shops could make almost any technology work and meet their needs.

Today, we see the same phenomenon, but in a different context. We see today that some shops are very successful in building, implementing, and maintaining a data warehouse. Other shops just can’t seem to get it right. And when they can’t get it right, they blame that “damned data warehouse.”

So the success factors – as in the old days – don’t really seem to have much to do with data warehousing. Instead – as in the old days – the success factors seem to have a lot to do with the organization implementing the data warehouse.

What do organizations that are successful do to enhance the opportunity for success in data warehousing? Organizations that have success almost uniformly:

Build their data warehouses in an iterative fashion. The “big bang” approach may work elsewhere, but not in the building of a data warehouse. There are actually lots of reasons for the need for iterative development; however, the strongest reason is that the end user cannot give requirements a priori for what the data warehouse needs to look like. The end user of the data warehouse operates in a mode of discovery. The end user needs to see what the possibilities are before the end user can say what is really needed. Therefore, it is guaranteed that the end user will change requirements before the final acceptance of the data warehouse.

Make sure that the data warehouse is built – from the very beginning – with the participation and the input from the end user. Building a data warehouse in a vacuum may produce a technological masterpiece which does nothing to help the end user community to make better decisions for the organization.

Build the data warehouse where the data is normalized and where the data is granular. The deep granularity provides some special characteristics to the data warehouse. Data with a deep level of granularity ensures that the analytical probes of the organization can be done to satisfy many different perspectives. With deep granular data, the organization can solve the information needs of the financial analyst, the marketing analyst, the sales analyst and the engineering analyst. In addition, with deeply granular data, the organization is prepared for future unknown analytical needs. Granularity is one of the hallmarks of the successful organization.

Prepare for large volumes of data. Successful organizations are aware that the data warehouse will bring with it volumes of data that have never before been seen in the organization. And with very large volumes of data come the issues of cost, management and access of data. Simply stated, management of gigabytes of data is one subject while management of terabytes of data is a different subject.

Manage metadata. Metadata is the glue that holds the data warehouse together. When looked at holistically, the data warehouse and the foundation for the support of the data warehouse is distributed. Without metadata, the various parts of the data warehouse are held together with chewing gum and a prayer. And for a professional organization, that just isn’t good enough.

So the next time somebody starts to complain about a data warehouse failure, remember that it is organization itself that is the most determining factor in whether a data warehouse succeeds or fails.

If you found this article helpful and would like to receive the latest insights each week from the experts featured on the BeyeNETWORK, please subscribe to the Bill Inmon Newsletter.