Blog: Dan E. Linstedt

« IT: Changing from Cost Center to Profit Center | Main | Is Modeling in your future? »

EII - Fight the Hype, Build EII for the Right Reasons!

Here we go again, YET ANOTHER EII vendor pushing the fact that they can "replace" the need for a data warehouse. In this blog I will talk about the issues that customers face if they DON'T implement a data warehouse. There are pros and cons to everything, fight the hype that EII is the be-all-end-all solution, it's NOT. EII is one successful piece to the puzzle; we just need to know where it fits.

The article is at: http://www.metamatrix.com/news/cbr-030705.pdf

The first quote I want to discuss is as follows:
"Proponents argue that EII replaces a physical extract of a data warehouse, thereby removing the need to spin-off expensive data marts."

Now just hold on, first off who says data marts are expensive? By who's measuring stick? Where are the numbers to back this up? Is this person discussing the entire corporate warehouse or just a single star schema?

Unfortunately we don't have answers to these questions, but if a vendor comes to you and pushes EII in this light, I would strongly suggest you ask them these questions. Let's dive in a little more.

EII Doesn't replace a physical extract unless the data that is wanted is current. It might be more purposeful to say that if EII replaced anything at all, it may be the need for the ODS itself, not necessarily the data marts. Data marts with history serve a huge strategic purpose. Integration systems (ETL/ELT) night after night, with the cleansing and integration of data on a massive scale serves a purpose. Most times the enterprise is looking for strategic answers across much of the history. EII is not built for TREND analysis, that's the job of the warehouse.

Now, can EII access the warehouse behind the scenes? YES! That's the beauty of the EII system, it can leverage the existing investment, it can also leverage all the knowledge and rules that have been built to integrate the data historically.

Here's the next quote:
“What’s the point of integrating a large amount of data when you only need a small slice of it?” Chappell asks. “Because the nature of corporate information is dynamic, trying to keep it replicated and synched in three or four databases when joined with another is impractical, especially if it’s accessed infrequently.”

The point here is interesting. Again, if all that is needed is OPERATINAL (Now) data with no history, EII IS a great solution. But if data has to be trended across history, it will need a data warehouse behind the scenes. Data Warehousing experts do not typically "integrate large amounts of data" for the fun of it. We have business reasons that derive value from all that history.

Who says we always replicate it in three or four databases? This is rarely the truth. It's like justifying their argument by saying: who needs three or four operational systems?

Here's the next quote:
“EII can help to ease the backlog of requested business reports,” says Chappell. “Changes to a data warehouse model to bring in new data can take months. EII isn’t as brittle as the procedural ETL scripts and can affect the necessary changes much more quickly.”

True, but only if the business reports are operational in nature. What this gentleman does NOT discuss is the impact on the operational systems of an EII query to originally GET the data out. There is a cost to using this technology, and it needs to be discussed. If you're talking to your sales reps for an EII solution, either bring in an expert to help with the evaluation, or ask the pertinent questions regarding impacts, standards, and best practices.

Finally, at long last, someone else in the article begins discussing the true value of EII as it pertains to the enterprise:
“EII isn’t an alternative to a data warehouse,” says Walcott. “Rather it augments historical time-series BI reporting with fresher operational detail.” He adds that EII is especially useful for situations where you want to get to detailed data that is usually omitted from the warehouse.

I agree - it definitely augments the reporting with a "fresh" view of the now or current data. EII is a powerful paradigm, allowing the strategic or historical batch warehouses to become an Active Data Warehouse overnight, without the higher costs associated with "activating" the warehouse by writing your own code to load it dynamically.

EII has tremendous value in the web-services production layer, along with the enterprise metadata management strategies. What we need to do as practitioners is figure out HOW to leverage the metadata across other tool sets, like retrofitting it (any changes to it) automatically into ETL/ELT, and propagating the business metadata into a metadata management tool for classification and ontology. One might say that another piece to the successful EII/ETL - Data Warehousing venue would be a service registry tool/utility, and a business metadata rules management component.

It saddens me to find that specific vendors are hyping the "cool-aid" approach to EII, this does nothing but damage the notions of EII and what it CAN do, damages the vendors (in the analysts eyes), and produces false expectations of the technology which IT cannot meet. EII is a GREAT resource, but should be utilized in the right light, and as an augmentation to the data warehouse in place, not as a "replacement" for future data marting or historical efforts.

Thoughts? Vignettes? Ideas? Pulitzer Prize winning essays?

Share: