Blog: Krish Krishnan

« September 2008 | Main | November 2008 »

October 29, 2008

Fraud detection - an interesting thought

It's almost Nov 4th and we have record voter turnout for early voting. While all of this is fine and dandy, I wonder if there is auditing in place on voters, when and where they voted etc, and are there checks and balances in place on ensuring no voter fraud occurs. If not, the next president will need to make that a US Government CIO's task to be implemented. We being a largely advanced nation in terms of technology cannot afford to be ignorant about this.

This election as such is historic enough already and we do not want additional attractions added to the same. The rest of the world is watching us.

Share:

October 27, 2008

A Data Warehouse Project is like ....

Running a long distance marathon. Well as a matter of fact, if you look at any data warehouse project, the key ingredient to the success of the project is the "endurance" of the team. Building a data warehouse is not a trivial task, you have people, politics and personalities to deal with, apart from tight deadlines, data issues like master data, metadata and data quality, business intelligence needs that surpass the capabilities of what is available, budget issues etc.

Managers often tend to overlook the "endurance" of the team. We often see projects fail due to nothing else but "burn outs". If you observe the marathon runners, they train differently for the marathon across their schedule. A data warehouse team is no different, they need to perform at different speeds at the various phases.

In a mad rush to keep deadlines and profitability, the data warehouse team is put under intense pressure, this leads to bad quality of work, which then results in a "tower of babel". Another key mistake that managers make unknowingly is display of authority, which is uncalled for. This may work in the military, but in real world, managers need to be more cognizant of facts and behave like coaches rather than disciplinarians.

I'm not trivializing the role of the manager in a data warehouse project, but hope that this will provide some insight from the trenches where the real work gets done. The manager of any data warehouse team does carry the brunt of responsibilities and is the customer facing representative, but sometimes situations tend to rule mind over matters and this note is to caution such situations.

Share:

October 19, 2008

Appliances Appliances Everywhere

Like the poem "water water everywhere", we seem to be inundated with appliances everywhere from every vendor. That is a lot of information to digest. In the coming months, to educate the users and decision makers alike on the technologies and which to choose and how to implement them, I'm teaching a course at TDWI New Orleans on Nov 7th, and will be holding an independent seminar series in three cities. More information on the three cities will be announced shortly. Keep checking my blogs.

Share:

October 14, 2008

Teradata - Data Warehouse Appliance

As I start this blog, I thank Teradata for inviting independent analysts like myself and others in the team to their Partners Conference at Las Vegas.

Teradata has announced the availability of flavors of data warehouse appliances. Yes you heard me correct, a data warehouse appliance. Wait a minute, Teradata had pioneered this concept years ago, so what is new? well if you think long enough, Teradata has identified itself to be an enterprise player and had always defined its vision to be the solution to the enterprise data warehouse. That vision is still strong and intact, but as an answer to the competition (In the kick off session on Monday, Mike Koehler CEO and Stephen Brobst CTO of Teradata mentioned this statement) in the area of the data warehouse appliance, Teradata now has a series of "purpose built platforms" that can provide scalable and robust solutions.

The customers of Teradata will love this, since it provides a single platform for consolidation. Infact Mike Koehler, the CEO of the company said that this year Teradata has been in head to head competition with other appliance vendors 30 times and had won 30 times, do not know how many times others won overall, I'm just writing what was announced yesterday.

What I see here is a pattern, this year first Microsoft, then Oracle and now Teradata, all of them bringing a data warehouse appliance platform within their product offerings, proves that data warehouse appliances are now mainstream.

From a customer base perspective and a product maturity perspective, there are leaders and visionaries. That is conversation for another day. I need to go to a "briefing" on the Teradata platform family in a few minutes and will add more to this blog at a later time this week.

Share:

October 8, 2008

A Conversation with Foster Hinshaw, Father of the Data Warehouse Appliance

Dear Readers,

In the spate of recent announcements from Oracle and HP about the Oracle-HP Database Machine and Exadata storage, I found some time to catch up with Foster Hinshaw, widely acknowledged as the Father of the Data Warehouse Appliance. Here are a few excerpts from our conversation:

Krish: Foster, in your opinion as the pioneer of data warehouse appliances, where does Oracle’s offering fit?
Foster: Krish, I have a very high regard for Larry Ellison and Oracle, but I feel that the Oracle is playing catch-up with the rest of the industry in this space. The technology is at least five-years-old and will need to mature fast.

Krish: Interesting observation. With the current growth rates in the industry, where is the real opportunity for data warehouse appliances in general and Dataupia in particular?
Foster: Well, the growth we are talking about is two fold, data is growing at 50% every year and users are growing by 30%. In this growth spurt, data warehouse appliances can offload the workload from the core database, thereby improving the overall system performance. Dataupia’s mission statement is “free your data,” and we believe in architecting solutions which will offer the users the scalability and flexibility to use their data from their data warehouse to achieve business performance.

Krish: Another key point that I have always advocated is the overall cost of the implementation. Your thoughts?
Foster: Yes, that is correct, whenever we talk about cost savings, we should account for the overall solution cost including implementation and ongoing maintenance. Our goal with Dataupia is to enable the scalability while maintaining the cost at a stable level from both the implementation and the infrastructure perspective.

Krish: One last question, is the Oracle-HP database machine truly MPP? In my opinion this is an underlying qualification to be considered an appliance. Whether MPP is in the software layer or the hardware layer, it still is the glue. How would you compare Dataupia and Oracle’s newest offering in this aspect?
Foster: Interesting opinion. Oracle’s new appliance is a solid first step toward full MPP. When we talk about MPP, we’re talking about data manipulation that is handled on multiple storage notes so that the user gets the benefit of complete data manipulation. At this stage, Oracle’s version is a light weight model of MPP in that it begins to put some, but not all, operations in parallel. Large data warehouse require extensive data movement and manipulation which is why Dataupia has and will continue to make MPP a core focus of their appliance.

If you are interested to see more excerpts from this discussion, please leave your comments and we will post more from this discussion.

Share:

October 6, 2008

DaaS - Data integration simplified

Look at the market, as I'm writing this blog, the DOW is below 10,000 points and there is a widespread fear on the global credit crisis. We (a few independent industry analysts) have been saying that BI is relatively on safe ground till this point. Now it is time to stop, take a deep breath and then start moving, that is this mantra that all companies are beginning to take as a step.

On Wall street we have seen the fall of major investment banks and the big brands have now been bought out or merged into other entities. The spate of M&A activities is not all said and done. With all these turbulent times, the need for a lower cost data integration service is becoming a thought that will cross many minds.

One of the ways to mitigate the risk while providing value is to implement Data as a Service (DaaS), to build a quick data integration layer. This approach while keeping cost down, will enable scalability and flexibility.

A survey conducted by B-EYE Network found that even after less than a year in the North American market, 23% of the people who responded were aware of DaaS specifically, and that 40% were aware of on-demand business analytics; 15% said they’d be willing to implement DaaS at their companies.

Why DaaS? there are several reasons and I will mention a couple

1. You cannot integrate all the data into your data warehouse overnight, but the Executives demand a dashboard that can show them data across the integrated enterprise.

2. There is data in the new merger or acquisition that needs to be verified and validated, and then transformed to integrate, but you do not have the hardware and the bandwidth to perform this activity.

One company that comes to my mind is Kognitio, which specifically deals with the DaaS solution set for the Data Warehouse. They have a plethora of options and offerings which can easily fit into the Insurance and Financial sectors specifically. You can check them out at their website "www.kognitio.com".

You can deploy the DaaS architecture internally or externally. My opinion is that large organizations will deploy it internally and smaller companies can use the external services. Data is an asset for the organization and hence the data security and compliance needs will need to be considered before sending the data externally.

DaaS is the way to go for data warehousing rather than the cloud.

There are other methods to achieve this kind of scalability with offerings from Dataupia, Vertica, ParAccel and AsterData, we will look at those in the next series of my blogs.

I have omitted the Oracle-HP database machine due to its newness to the market.

Teradata, Oracle, DB2 and SQL Server are all in all these organizations already and the key issue is to integrate them with ease. These are source systems and enterprise platforms which are the underlying infrastructure today.

Share: