Blog: Dan E. Linstedt

« How Data Models can Impact Business | Main | Hardware: What do you need for VLDW? »

Commoditization of the EDW/BI Market.

Like all markets, the EDW and BI market is moving (if it hasn't already) towards commoditization in implementation. Many would argue (even today) that this is "impossible" - as I've heard it, they still believe that every DW / BI solution must be custom driven, must be custom loaded, must be custom created. Why? Because we have "different requirements/different data/high volumes/real-time" and so on. We've heard it all before, it was the calm before the storm when ERP vendors rose up and said: You CAN have a standard, and the standard is "our solution." It happened with Source Systems, and now it will happen with EDW/BI solutions.

Wait just a minute, you mean to say that we are or will all be building EDW / BI solutions the same way every time?
Yes. It's the nature of any industry: first, we figure out how to do it, then we all develop "specialized ways" of doing it, then someone begins to use standards to build the solutions, then someone else says: with standards, we can repeat it. Then we "hopefully" optimize the standard process for building it.

Finally, someone says: here's a solution wrapped up for you that is optimized, and handles your line of business according to the standards.

All we have to do is look around at the industry.

1) We have a standard over-arching framework for I.T.: Zachman Framework.
2) We have a standard frame-work for EDW/BI: DW2.0, CIF, GIF
3) We have a standard logical data model: Universal Data Models
4) We have a standard data modeling paradigm: Data Vault Modeling Architecture
5) We have repeatable, consistent, redundant, and reliable business mechanisms to get things done: ITIL, CoBIT, SEI/CMMI principles and so on.
6) We have standard local and global financial principles to account for: G.A.A.P, and SARBOX, and other compliance initiatives

Now, do we have the tools or the pre-packaged solutions available?
Well, we have some vendors in this space that are starting to bring this to the table, we have others that have _highly_ specialized solutions to solve specific problems (not standard enough to be wide-spread).

Well, some would say vendors tried this in the past with "Analytic Applications", aren't we returning to that path?
No, analytic applications from a vendor perspective cost too much to customize, didn't serve all the data from all the source systems that existed, couldn't be repeatable, redundant, or fault-tolerant (once modified out of the box, they lost their value - the vendors couldn't upgrade them without destroying the customizations).

In order for commoditization to happen, someone needs to wrap business functions with EDW with BI and meet all the above standards, while providing a flexible, sustainable architecture under the covers. That someone also needs to provide a full-solution (very large, like SAP did when they first entered the market place), and it needs to be a HORIZONTALLY posititioned solution rather than a vertical stove-piped solution.

Will it happen?
You bet, it's happening now (still on a manual basis) but there are those of us in the industry who can / and have built standard solutions in a commoditized fashion for several different companies.

So what, what's the big benefit of commoditization?
Well, since you asked: LOWER COST. It's incredible how much lower the cost can be when the EDW/BI solution is repeatable, consistent, fault-tolerant, and reliable. An entire EDW that shows the business results from a full source system can be built with BI in 1 to two months now. Most people think of 90-day time-box deliverables for a single scoped portion of the business, but I see a trend, downward to 1 month deliverables - and with less people to build it as well.

Commoditization will happen within the next year, mark my words - no literally, book-mark my words... (just kidding). It will happen, and the first movers will gain much of the competitive advantage.

Typical signs of Commoditization:
1. fully-configured appliance, installed and running in record time.
2. 1 consultant on-site instead of a team of 5 or 10 people
3. Full source system integrated within 2 to 3 weeks, and business reports within 1 to 2 additional weeks (provided the business turns around their requirements that fast).
4. Generated load routines
5. Consistent, repeatable, measurable business process for EDW/BI
6. Optimizable business process
7. Greatly reduced risk to build out
8. Incredible cost savings (instead of $250k for a prototype, cost is $50k to $75k)
9. I.T. becomes extremely nimble and adaptable to change in the EDW / BI environment, extreme agility.

Do you have a project that was done like this? I'd love to hear from you.
Dan Linstedt

Share: