Blog: William McKnight

Main

October 5, 2007

Column-oriented makes a comeback... with the DW appliance model

Oops, what is that I tripped on? Oh, it’s another new column-oriented data warehouse appliance. If you haven’t noticed, in 2007, Vertica, ParAccel and Calpont have emerged with a column orientation to their DBMS and the appliance model to their delivery. By the way, that makes 12 data warehouse appliances by my count.

A phrase I saw on the internet recently - “Pioneer calls RDBMS technology obsolete” - caught my eye and the first thing that came to mind was “Michael Stonebraker?”. My suspicions were correct. Vertica is his new venture and he states “my prediction is that column stores will take over the warehouse market over time, completely displacing row stores”.

Most IS professionals do not know about column (or “vector”) oriented DBMS. Column-oriented DBMS have several major architectural differences from other relational database management systems. The main difference is its physical orientation of data in columns as opposed to rows. This allows it to perform very high selective compression because all of a column’s values are physically together. It also provides for excellent performance when you select a small subset of the columns in a table since you do not perform I/O for data that is not needed. Column-orientation greatly assists a compression strategy due to the high potential for the existence of similar values in columns of adjacent rows in the table.

The Model 204 was sort of like this and Sybase IQ is definitely column oriented. There have been special occassions where they are more appropriate than the row-oriented DBMS.

It will be interesting to see where and how these approaches find merit in DW, if they have overcome some of the problems of the past such as those below (early indications are that they may have) and finally, if they intend to compete for EDW, as Michael Stonebraker suggests in his quote above.

Continue reading "Column-oriented makes a comeback... with the DW appliance model" »

Posted by William McKnight at 8:26 AM | Permalink | Comments (0)

October 18, 2006

Consequences of making inappropriate DBMS selection for Data Warehousing

Some of the consequences of making inappropriate DBMS selection for DW/BI include:

• Long development cycles
• High numbers of support staff required
• Cost expansion
• “Throwing hardware at problems” as a solution
• Users reverting to old means of data access with user interfaces that are not friendly
• A technology-focused culture rather than a user culture in IT
• Complex vendor relationships
• Hard to incorporate legacy systems and unstructured data
• Inability to keep pace with growing data volumes and user demands
• Inability to show profitability from data warehouse efforts, leading to slow program demise

Posted by William McKnight at 11:50 AM | Permalink | Comments (2)

October 12, 2006

Criteria for a Data Warehouse DBMS Selection

Based on the realities of data warehousing today, a selected technical architecture for data warehousing should be:

• Manageable - Through minimal support tasks requiring DBA/System Administrator intervention. It should provide a single point of control to simplify system administration. You should be able to create and implement new tables and indexes at will.
• Complete and Integrated – The toolset should be comprehensive across the spectrum of eventual requirements for data and its access.
• Interoperable - Integrated access to the web, Microsoft Office, internal networks, and corporate mainframes.
• Scalable – The ability to handle increasing data sizes and workloads with simple additions to the architecture, as opposed to the increases requiring a rearchitecture
• Affordable – Proposed solution (hardware, software, services, required customer support) providing a low total cost of ownership (TCO) over a multi-year period.
• Proven and Supported – You don’t want to risk a critical decision regarding a fundamental underpinning of the data warehouse environment on an unproven solution.
• Flexible - Provides optimal performance across the full range of models with large numbers of tables. Look for proven ability to support multiple applications from different business units, leveraging data that is integrated across business functions and subject areas.
• User Accessible – Compatibilities and interoperability with data access tools that provide a wide range of user-friendly access options.

Posted by William McKnight at 8:47 AM | Permalink | Comments (0)

September 7, 2006

Gartner Magic Quadrant for Data Warehouse DBMS Servers, 2006 is out

The Gartner Magic Quadrant for Data Warehouse DBMS Servers, 2006 is out now and available at link. As the report itself says "NCR's Teradata server is the clear leader."

This report is just for the backend server used to house the data warehouse. And, of course, you never buy just that. You would also need to be comfortable with the DBMS possibilities for that server. For example, Teradata DBMS on Teradata. Initial cost, lack of openness and lack of low-end MPP mean that their placement does not directly translate to market share.

Interestingly, at the other end of the Ability to Execute spectrum is Netezza, which should gain acceptance as their customer base grows and Linux gains acceptance, both of which are happening.

These are the only 2 on the list where a specific and single DBMS is tied to the platform. Most companies start with the DBMS in their selection process of DW platform.

Posted by William McKnight at 1:22 PM | Permalink | Comments (1)

June 8, 2006

RDBMS market share numbers 2005

IDC's RDBMS 2005 Vendor Share report came out last month. Here's a link to it on Oracle's site. Oracle has good reason to post it on their site. They still dominate the market - by far.

Microsoft is making huge strides and of course, IBM is in the mix. In the 'Future Outlook' of the report, there is a note about open source RDBMS such as MySQL. I would also throw in data warehouse appliances and suggest that these are 2 emerging technologies that are potentially disruptive to the status quo.

When market share is measured for 2006, I expect further increases from Oracle, Teradata and Microsoft. Perhaps by the 2007 numbers, Netezza and MySQL will merit their name on the list.

Posted by William McKnight at 8:19 AM | Permalink | Comments (3)