Blog: Krish Krishnan

« A few things needed in any Data Warehouse Appliance | Main | Number Crunching Cost Effectively »

Why an Analytical Data Warehouse Appliance

We all have been discussing, writing and reading about Data Warehouse Appliances. Still in the formative years, this technology is already making rounds in the data center. Recently I had the opportunity to spend time on columnar databases based Appliances. This technology is just awesome for Analytical applications. Why do we need a separate appliance for Analytical purposes.

When we look at the current RDBMS technologies, they are geared towards OLTP applications. They all provide analytical functions built on the database platform. But these functions when running on the traditional SMP architecture cannot perform at high speeds and encounter severe disk and cpu constraints. This is especially a fact when it comes to running OLAP queries on the SMP architecture.

This is where a columnar database differs. Traditional databases processes queries in a row based fashion while columnar databases processes queries in a columnar fashion. The architecture of a columnar database is

1. Columnar data storage
2. High-performance data loads and updates processing
3. Shared-nothing and Massively Parallel Processing architecture
4. Adaptive compression of data based on data type and length
5. Data is stored compressed on the disk

The query processing architecture of a columnar database is

1. Only columns relevant to the query being executed are retrieved
2. All operations are done in parallel (A traditional DBMS will scan all of the data sequentially)
3. There is a very low overhead in data retrieval
4. Data is scanned compressed and only expanded on retrieval.

Since the underlying database is architected to store data compressed and retrieve data in columns rather than rows, there is an advantage in building a multidimensional query on this platform. While there is a potential for the columnar database to provide a platform advantage for the Analytical data warehouse, the other appliance technologies also provide a similar advantage in terms of performance.

One columnar database vendor has already proven their database strength by executing the TPC-H benchmarks.

As Operational BI matures over the next few years and the demand for operational reporting increases there will be an increase in demand for data availability and data accessibility. These are the technologies that will be deployed in the data center to augment the workload from the data warehouse.

Watch this Blog for further details on this topic.

Share: