Blog: Krish Krishnan

« May 2008 | Main | July 2008 »

June 29, 2008

Number Crunching

A concept like none other, number crunching has always been thought of in scientific applications, real time applications and complex processing of large numeric calculations. With the rebirth of Data Mining and Predictive Analytics and the increasing adoption to Data Warehousing and Operational Data, number crunching will soon be common to the data practitioners, if not already.

How will number crunching matter to business intelligence and data warehouse professionals? think of it in terms of data itself, instead of 3 years of data, if you have access to 10 years or 30, years of data, the impact will be quite a splash. From a solution perspective, there are multiple layers of impact from the front end tools and infrastructure to the back end tools and infrastructure.

The cost of infrastructure will be a non issue with the advent of data warehouse appliances from a back end perspective. But the cost of front end tools and processing of the result sets will be expensive, this is due to the cost of the algorithms and the analysis required from the results. Over a period of time, with more standard statistical models, this cost can be commoditized.

What is the advantage of providing a number crunching platform for data warehousing? the advantages are manifold. More data will mean better insights to the subject area under analysis
Larger data sets will provide a robust platform for analytical applications like SAS and KXEN.

Another topic that is creating a lot of excitement is integrating unstructured and structured data in the data warehouse. This subject is even more exciting when you think of a medical software like ISABEL used across hospitals getting an integration with staff notes from years of patient data. Such an integration will be loved by both doctors and insurers alike.

As we move in to the new era of data warehousing, let us also embrace the concept of number crunching into this side of the world.

Share:

June 25, 2008

The Appliance Gets Mainstream

With the announcement from HP yesterday that read "Hewlett-Packard (NYSE: HPQ) on Tuesday introduced a BladeSystem server configured for an Oracle data warehouse.

The BladeSystem for Oracle (NSDQ: ORCL) Optimized Warehouse is a pre-configured appliance-like offering for the 1- to 4-TB market. The HP system provides the essential elements of a data warehouse infrastructure, including computing power, storage, interconnect, and management.", the data warehouse appliance gets mainstream attention.

What remains still an attraction to the non traditional vendors like Dataupia, Paraccel and AsterData is the fact that these companies are focused on solving a specific need from the data warehouse perspective, while the big RDBMS vendors are attempting an extension on the current platform.

With the cost of storing data becoming cheaper, organizations are looking to scalable but economical solutions, whichever way we go, the market is beginning to wake up and this is great news.

Share:

June 16, 2008

How to defeat a successful data warehouse program

Wow, this sounds stupid right. how can someone not want a successful data warehouse program? Guess what, while it is unbelievable it is true that even without trying, this is the fact across organizations. What makes this happen? well it my favorite subject and talking point, organization politics.

How does organization politics affect the project? well if the communication between different teams in a project is not clear enough and there is no executive leader who charges the team with responsibility and empower all the players appropriately, the end game is what a dear friend of mine calls "goat rodeo".

This exactly is the mantra on "How to defeat a successful data warehouse program". Why do organizations get this behavior? No one answer can be given to this question. But how do you change this behavior and get the data warehouse program to be a very successful program. This is where the key points as discussed below come into play

1. Executive Leadership - A strong executive leader is needed to lead a very successful data warehouse program. This leader should be empowered to make executive decisions on the program in the organization.

2. Empowered Teams - A SWAT team of players in each stakeholder team is needed to make decisions on the program. The teams should be from IT, IS and Business areas respectively.

3. Steering Committee - A steering committee of the executive leader and the SWAT teams will be a core team to guide and move this program.

4. Supporting partners - A complementing team of supporting partners is very essential to enable the core team to implement the program.

5. Policies and Procedures - A clear set of rules of engagement will be needed to be established and followed to implement a successful program.

6. TEAM - A very important concept, once you form a core team to start the data warehouse program, the word TEAM is very important. Often human "ego" plays a very negative influence in the success of a data warehouse program. Though a large number of the readers and colleagues may oppose this point, it is a hard fact that often a clash of "egos" leads to a traumatic effect on performance of the TEAM. A gentle but firm suggestion is to ensure that a data warehouse team be advised to be passionate about their work and program and not be overcome by emotional or egoistic issues which leaves a bad flavor on everybody in the organization.

Once again I hope that "we" all as data warehouse professionals ponder on this subject and provide ourselves and our organizations with some introspection on this subject, to ensure that data warehouse programs have a high success ration forever.

Share:

June 5, 2008

Data Visualization - A critical success factor

Data visualization is a very essential component to ensure the success of a Business Intelligence and Data Warehousing project. While this is not a first time writing on the topic, it is essential to point out that this fact is often overlooked when it comes to testing and in particular user acceptance areas in a BI project.

Why is visualization important? well we all know that a picture is worth a thousand words. When you are able to see a data graph or a tabular report from a BI project, you can readily see the effectiveness of the new system from both a data accuracy and a query performance perspective.
This becomes critical as the complexity of the system increases and the complexity of the visualization increases.

Visualization also enables user adoption and executive sponsorship participation. Both of these components are highly critical for a successful BI project. The market today is being flooded with new data visualization engines, some of which can run data from an ODS for operational reporting purposes, out of the box, with minimal customization.

To ensure success of any data warehousing project, when possible, use the visualization technique and you will be well satisfied with the outcome of the project.

Share: