We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Expanding the Envelope

Originally published April 24, 2008

Data warehousing is not a new topic. Companies have been designing and deploying data warehouses for decades now. Recently, there has been a renewed level of interest in data warehousing, but not just traditional enterprise data warehousing (EDW). Customers are now in need of massive storage capacity to accommodate dramatic increases in their data. Previous online transaction processing (OLTP) database architectures don’t meet customer needs, and EDW prices are too high for the new volumes of data. Database companies are rethinking their designs. What makes an old design concept new? We find ourselves in a renaissance of thinking about data warehouse architectures and luckily for all of us, there is no shortage of high quality engineering combined with venture capital to fund the plethora of companies bringing many new ideas to market. Look around and you will see innovative data warehouse appliances, columnar databases, new designs incorporating Hadoop, and an enormous increase in speed and scale in the data warehousing world compared to even five years ago.

What do these developments mean to customers? I see two ways in which this is advantageous for consumers of data warehousing:

  1. More commercial alternatives from which to choose a technical solution.

  2. Lower prices as companies compete for data warehouse projects.

And as data volumes are starting to grow rapidly into hundreds of terabytes and into the petabyte range, we find that customers are pushing the limits of what is possible. Not just Fortune 100 customers, but small Internet start-ups and mid-tier financial services companies as well. We see much lower cost hardware and software solutions emerging that enable these companies, from start-up to global enterprises, to now store and process massive amounts of data. What is happening in the data warehouse companies that makes this possible?

The aviation industry provides a metaphor. This is an industry that has been able to grow the range of ways you can transport people and cargo over the last century through a process known as “expanding the envelope.” As a design for a particular aircraft model becomes stable, engineers consider key design criteria in a kind of two-dimensional chart known as an envelope, and they choose to selectively expand the envelope along one or another dimension. They evaluate pros and cons of each design choice knowing that the right answer is not always to push in all directions at once. They push the envelope with test flights using actual planes and incrementally discover what they need to improve in their engineering designs in order to reach the next level of performance. Although some steps seem small, when taken in aggregate, the resulting improvements are magnified. Over time, great innovations are developed, carefully put into test and then put into production. The rate of change is steady and unrelenting, motivated by competition and by customer demand. The range of choices in air transportation today is orders of magnitude better than that offered in the past, and this trend will continue. As companies compete by continually expanding the envelope, customers win.

As an example of a data warehouse company that is expanding the envelope, Luke Lonergan, Greenplum’s CTO, has commented that this is how his engineering organization is making strides in the development of massive data warehousing. Like all the companies in data warehousing, no single solution makes sense for every possible use case. Some choose to focus on speed but trade off flexibility. Others focus on reliability but trade off cost. What suits your need will depend on how you choose to “expand the envelope” within your organization, be it a competitive advantage you seek to exploit with your data warehouse or a cost advantage you extract from your customer data. Whatever the measure, I see this as a win/win situation for the providers of data warehousing solutions as well as the end users of these solutions. Look to the data warehouse vendors to see where they are choosing to expand the envelope. This will provide you with the kind of clues you need to determine what is the right solution for your particular data processing requirements.

  • Bill InmonBill Inmon

    Bill is universally recognized as the father of the data warehouse. He has more than 36 years of database technology management experience and data warehouse design expertise. He has published more than 40 books and 1,000 articles on data warehousing and data management, and his books have been translated into nine languages. He is known globally for his data warehouse development seminars and has been a keynote speaker for many major computing associations.

    Editor's Note: More articles, resources and events are available in Bill's BeyeNETWORK Expert Channel. Be sure to visit today!

Recent articles by Bill Inmon

 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!