Leave it to Microsoft to make technology inexpensive and easy. Quite possibly (i.e., link) Office 14 will begin to include the Stratature Master Data Management (MDM) acquisiton. Microsoft has helped to demystify data warehousing and data mining and now begins to do the same with MDM.

For the MDM market, this is a partial verification and will expose MDM to a wider audience. Those who are opposed to Microsoft philosophically or don't believe it will scale will go elsewise for their tool.

Tools are a start, but can obfuscate the need to address data quality, still the largest impediment to information management success.

End clients generally end up overrating the usability of packaged data models, whether as stand-alone models or those models inherent in their software packages. Sooner or later, some amount of what I call ‘original data modeling’ is going to be necessary in any enterprise. Modeling expertise is a must-have though companies go to enormous lengths sometimes to avoid it. I don’t advocate necessarily changing packaged models in applications. However, no untouched application models, or combinations thereof, are going to be completely sufficient for a data warehouse in Fortune clients.

Time for the Wednesday (or thereabouts) “What” (what I have learned…). OK, I seem to be endlessly prompted in my client work with these learnings so there’s no shortage of them, but sometimes I don’t have an elegant preamble to a blog entry. So, I’ll just say it.

You’ve got to tie that warehouse data back to source or users will cry foul. It doesn’t matter how dirty the source data is. If you want to change the data en route to the warehouse to clean it, fine, change it, but bring the original data as well in a different set of columns in order to prove your tie-out.

Tie-out should make you more comfortable with your ETL as well. It sometimes involves adding pre-extract queries to the source data and post-load queries to the warehouse data. It sometimes involves ‘spot’ query checks, which can get tricky. I.e., the method used to pick your spot data can come under scrutiny. It also gets tricky when the ETL is run intra-day or real-time, when ETL cycles are at an absolute premium. However, you still need to do it IMO. These tie-out results go in your operational metadata.

Here's another "what I've learned." When we have a deadline or a business-critical issue, we can tend to underemphasize the business requirements, due diligence, experience and sometimes some spending that is going to be necessary.

One area, just to pick something out, is Dun&Bradstreet numbers. DNB is practically ubiquitous in my clientele yet it’s always harder to match than is initially believed. Another truth about DNB is that not every company has a DNB number. Plan for it!

I have a collection of data warehousing “famous last words.” Here’s one: “We’ll just link our customers to DNB tomorrow and it’ll be ready for you on Thursday.”

Here’s one more on the theme of what I’ve learned. You’ve heard of Don’t Mess with Texas? Well, how about Don’t Mess with Excel!? Users love the sense of control over the data and the ability to perform their own calculations. (Other) BI tools will not sweep Excel out of any enterprise. However, precisely because of its flexibility, Excel is notorious as a source system for data warehousing and its applications need to get into a DBMS to serve the organization in that capacity.