Blog: Krish Krishnan

« December 2007 | Main | February 2008 »

January 30, 2008

IBM acquires another one

The consolidation spree has not yet stopped. A week ago IBM announced the acquisition of AptSoft. What is the driver behind this acquisition?.

AptSoft platform provides a Complex Event Processing (CEP) design and execution engine to address real-time, event-driven applications characterized by the ability to detect patterns of events occurring over different time periods across the extended enterprise IT infrastructure and responding by intelligently orchestrating system and human activities.

What are some of the characteristics of these applications
* They are driven by the occurrence of key events or patterns of events
* They are agnostic to the time and order of events or patterns
* They are Non-linear i.e an event occurrence and its responses are unpredictable and vary by each occurence
* They are dynamic, meaning the underlying business processing logic changes often.

Typical applications range from online retailers to fraud detection to anti mony laundering systems.

An interesting point in this is the application of this technology into your ODS and DW implementations. Now with an SOA ability to watch the data flow across the operational systems and interrogate the ODS with pattern searches dynamically, an new dimension to operational reporting is possible.

Another area where this application will provide enterprise data integration a boost is CRM.

While all of this is possible in AptSoft today, IBM's vision for the future of this integration remains to be seen. Will DB2 in a future generation have event (SOA) driven data warehouses? we do not know the answer today but it is a possibility.

Share:

January 27, 2008

IS this the Year of Data Warehouse Appliance

IS this the year of the Data Warehouse Appliance. With the consolidations in the technology sector still going strong, the opportunity for the new and young companies in the Data Warehouse and Business Intelligence space is emerging stronger. The new offerings will need platforms based on commodity hardware and storage offerings. This is where I see a huge opportunity for the Data Warehouse Appliance.

The market is agog with making the Data Warehouse Appliance a modular out of the box offering that can be bundled in a rack and stack configuration with solutions like MDM or CDI built on top of the appliance. But there is more to this offering than just a modular component that it may get reduced to.

The technology is young and needs more understanding and adoption than what is perceived. The road is long but the journey is not hard. Data Warehouse Appliances by their nature of construction offer high performance and can handle large volumes, by now at least this is a clear point.

With so many competing offerings, which one lends itself to your enterprise architecture depends on your current investment and future needs. With this as a base statement, when you start looking at the technology platform for your enterprise in the data warehouse and business intelligence areas, the appliance provides options that certainly open a whole new world.

While the appliance by itself is not a solution for a BI problem like MDM or CDI or any application, it is emerging as a strong platform for the infrastructure and is being extended beyond just solving Data Warehouse and Business Intelligence needs. IS this the year of the Data Warehouse Appliance?.

In the coming month please see the BEyeNetwork.com site for white papers and research on applying the Data Warehouse Appliance in and outside of the Data Warehouse.

Share:

January 24, 2008

Why do you need Organizational Alignment

What makes a data warehouse or a business intelligence initiative a successful one?. The answer is simple, an informed and aligned organization. Yes it is a true fact that when you have organizational alignment on a new initiative like your DW/BI project or program it is more simpler to manage since there is a pride of ownership and all the stakeholders participate equally in the process and thus the overall initiative is a resounding success.

How do you go about getting organizational alignment. Whether you are a SMB with a DW project or a large fortune 50 company, you need the following individuals or teams to be supportive and involved in a DW project/program

1. Business Sponsor - This is a team or an executive who champions the initiative in the organization.
2. Business Users - This is a team of end users who will be the customers of the DW project. The alignment of these users into the project is a critical success factor.
3. IS teams - This is a team that develops the DW and BI projects and provides all the technology support to develop and maintain the DW.
4. QA teams - This is a team of blended business analysts and quality assurance analysts who can validate the DW and BI projects to ensure accuracy and quality.
5. IT teams - The infrastructure team that maintains the hardware, storage and supports the technical infrastructure is a key component to the success of this program or project.

When you have these teams in alignment on your business requirements, technology selection and implementation schedule, you can be assured of success in the DW initiative.

To build out the DW/BI projects you still need methodology, management, design, budget etc, but all these items rely on the teams coordinating in the initiative. There are a number of books and articles on this subject to read, but simply put if you can follow simple rules and achieve organizational alignment, more than half your battle is won.

Share:

January 16, 2008

Another new for Microsoft - CRM at Grocery Supermarket

In the near future, you will be grocery shopping at the local supermarket and your smart-cart (not to be confused with the smartcarte at the airports) will keep displaying coupons for the items on sale for that week in the aisle you are currently shopping. We are not talking about a Tom Cruise or a Spielberg movie here, this is in the news (on national TV) and the company working on integrating RFID to your grocery cart and provide real-time CRM is Microsoft.

Even if this is in the works and is a couple of years away, with windows mobile and RFID and a database (maybe SQL Server 2010 or something like that), the cost of development and implementation notwithstanding, it will open a new channel for DataWarehousing.

We are talking about Retail Data warehousing at your grocery store, doing real time market basket analysis, developing an Amazon like clickstream display showing based on sale items, what other items customers have bought in the store. Plus it will give the store operations and labor management real time data to analyze and improve operations and staffing and predict better product movement. Combine things like weather and seasonal data, you can do trend analysis and predict sales. A chain like Safeway stores can now think of store, district, regional and national analytical engines for product sales that will work close to real time.

Whew, real time CRM, Data Warehousing, MDM, Analytics all these get a whole new lifecycle to look forward too in a new vertical whenever this happens in the future.

Share:

January 8, 2008

Number Crunching Cost Effectively

Hello and Happy New Year 2008.

In my last blog, I started out on discussing why we need an Analytical Data Warehouse and mentioned about Columnar Databases. Continuing on that thought, whenever we talk about Predictive Analytics and Data Mining exercises, we all think of the PhD analysts and the complex algorithms they write, and the massive volumes of data they ask for before they come back with thier models and analysis. Bottom line, when looking at these exercises even today they are expensive and complicated. We cannot replace the statisticians since they are the SME's, but we certainly can look at providing them better infrastructure and accelarate the analysis time.

Statistical models or Analytical models as we all know it, take a large multidimensional dataset and analyse the same through complex calculations to arrive at results. A traditional database engine can accomodate this requirement, but we run into the same old OLTP bottlenecks on shared everything database execution causing CPU contention, heavy IO and slower network bandwidth.

While we get results, they are not quickly available as desired. If the results are needed quickly then the infrastructure costs go through the roof. Due to these catch-22 issues many organizations often end up dismantling their Analytical and Statistical BI initiatives.

This is where once again I see a future for the columnar database. due to this technology, we can compress the huge multideimensional data and leverage all the benefitsthat is available from this database technology. On this note I also point out that Data Warehouse Appliances can provide an alternate platform to implement this database.

By implementing the analytical database on a columnar database technology, the application server for the analytical database can offload all the heavy and complex calculation to the database engine.

While we all may be skeptical about the whole concept, I can visualize the merits of this solution and the underlying blueprint that it can provide from a platform perspective.

From the next blog series that I will be contributing, we will continue to explore industry specific issues and share the situations faced in the field everyday in different disciplines of BI.

Share: