Blog: Dan E. Linstedt Subscribe to this blog's RSS feed!

Dan Linstedt

Bill Inmon has given me this wonderful opportunity to blog on his behalf. I like to cover everything from DW2.0 to integration to data modeling, including ETL/ELT, SOA, Master Data Management, Unstructured Data, DW and BI. Currently I am working on ways to create dynamic data warehouses, push-button architectures, and automated generation of common data models. You can find me at Denver University where I participate on an academic advisory board for Masters Students in I.T. I can't wait to hear from you in the comments of my blog entries. Thank-you, and all the best; Dan Linstedt,

About the author >

Cofounder of Genesee Academy, RapidACE, and, Daniel Linstedt is an internationally known expert in data warehousing, business intelligence, analytics, very large data warehousing (VLDW), OLTP and performance and tuning. He has been the lead technical architect on enterprise-wide data warehouse projects and refinements for many Fortune 500 companies. Linstedt is an instructor of The Data Warehousing Institute and a featured speaker at industry events. He is a Certified DW2.0 Architect. He has worked with companies including: IBM, Informatica, Ipedo, X-Aware, Netezza, Microsoft, Oracle, Silver Creek Systems, and Teradata.  He is trained in SEI / CMMi Level 5, and is the inventor of The Matrix Methodology, and the Data Vault Data modeling architecture. He has built expert training courses, and trained hundreds of industry professionals, and is the voice of Bill Inmons' Blog on

Every good BI/EDW solution is backed by a good architecture, DW2.0 is no different. The frame-work that DW2.0 provides is a sound framework with all the components necessary. That said, in addition to the framework, architectures need to exist at different levels, as do standards, and templates. A solid enterprise data warehouse project usually contains many of the following components that implementers and consultants use to make a project successful.

The components include:
1. Common Project Plan (usually around 1400 to 2200 line items)
2. Business Requirements Document
3. Technical Requirements Document
4. Work Breakdown Structure (WBS)
5. Data Breakdown Structure (DBS)
6. Organizational Breakdown Structure (OBS)
7. Process Breakdown Structure (PBS)
8. Skills Assessment Matrix
9. Risk Analysis Document (high level)
10. Risk Analysis documents for critical tasks
11. Critical Process Flow Definitions (as-is), both business and technical
12. As-is data model documents
13. To-Be (proposed) E-R diagrams for staging, EDW, and at least the first mart structure
14. To-Be (proposed) interface structures
15. Cross-Reference (from/to) table.element to table.element definition (source to target design spec with transformations)
16. Standards Templates for loading CLASSIFICATIONS of table structures.
17. Reporting Templates for providing data - usually part of the business requirements document, that is: existing reports they may be receiving. Sometimes the To-Be reports are defined in the technical requirements document. However, reporting templates should be designed according to the GRAIN of data they provide, along with the OBS that they serve.

This is not a comprehensive list, but it has the major components that are necessary in all projects. Keep in mind that a lot of these items can usually be purchased as templates from ITIL (the IT Information Library), or ISACA. Some of these things are available for purchase from SOME consulting companies.

I, being of sound mind (although some would beg to differ) hereby offer up (within this series) documents that might assist you along the way. I'll post these as we proceed on my companies' web site. Ok - enough of this... back to the topic at hand.

What are the key components?
1. It is vital, absolutely vital to put all of these (and other project artifacts) under a single, company wide version control.
2. ALL EMPLOYEES DOWN TO THE LAST BUSINESS WORKER SHOULD HAVE ACCESS TO THE ENTIRE STACK OF DOCUMENTS AT ANY TIME, that is of course unless your running a cleared/classified project. All these documents should be available for read-only access on the web, by any member of the business or technical side of the house.
3. ALL documents require sign-off sheets for changes, and adjustments - these sheets are signed and dated by the money holder, project sponsor, technical project lead, and project manager. If you have a PMO office, they should handle SOME of this documentation, it is unlikely they handle it all unless they are SEI / CMMI Level 5 sanctioned.
4. ALL documents must contain technical numbering. It starts with the Business Requirements, followed by the tech requirements, followed by the project plan.

for instance: in my Busn. Req. doc, should match in my Tech requirements doc (where we discuss the implementation aspects), which should match in my project plan - the resource load, progress tracking, and dependencies to getting it done. The title in the project plan should be a tech requirements title, major numbers like 2.0 should be a business requirements doc paragraph title. MIX the metadata!!!

Oh it get's better.... should also match the WBS - telling me what work needs to be done to accomplish the tasks identified. should also match the OBS - telling me which roles & responsibilities (which are usually loaded in to project plans) are needed, and what the escalation paths are in the organization should something go wrong. It also attaches the sign-off document to individual components. should also match the PBS - which define the current AS-IS processes that are involved in currently getting the work done, which of course also define the critical path elements (business process management/re-engineering) so that the TO-BE process designs are more efficient. should also match the DBS - which defines what data is needed to meet the requirements.

In other words, the project line-item number should be pervasively applied to every document in the stack, so that monitoring project progression and completeness can be done. You can't reach Level 5 CMMI without monitoring, and automating (where possible), and you can't fix what's broken unless you can plainly see the hold-ups, delays, and large issues in the way.

WHAT??? There's no possible way I can get all of this for the whole project before I start... that would be MASSIVE effort!!
Yes and no.... it's all about scope. Make absolutely certain that you scope down what you are implementing, and scope down the documents as needed. The only documents which usually don't scope down are the project plan, and the business requirements documents. Once you have these, scoping is much easier.

Remember: that the size of your implementation team can bear a huge impact on the scope you can accomplish and the time you can accomplish that scope in. In other words, it can contribute to overall project RISK to not have enough resources. It needs to be identified in order to discuss with the business before moving forward.

Now, that said: I speak from experience, as a technical project manager in a HUGE organization, I once went from zero to all documents in place, and the project phased and then scoped within 3 months. Also within that 3 months my technical team of 3 - 4 people went from zero to delivery of a single report/single star schema. It is possible. Sometimes this is "sold" as a 90-day time box deliverable, but be-wary of the deliverables that do not include the methodology, the approach, and are missing the documents I specified above.

Why? because when the consulting company finishes the work, and turn over the efforts, if you don't receive the methodology behind it, and the documents aren't numbered, then it is VERY hard for you (the customer) to learn about what you have, much less maintain it going forward.

Usually business requirements documents are huge, in order to meet all the items within the BPR, it is absolutely vital to divide it in to phases (I like a 3 phased approach). This way, eventually all the requirements can be met, and in the first phase some low-hanging fruit (like necessary infrastructure) can be built along side an expandable and flexible architecture also along side NECESSARY requirements that are on-fire.

LET THE USERS WORK OUT WHAT'S HIGH PRIORITY FOR PHASE 1, once they understand they _will_ eventually get all requirements, just not in the first phase, they usually settle in to the task at hand of assisting to define scope WITH YOU. Ensure that you help or assist them to define scope of the project as you move forward.

As usual, I'm interested in your thoughts and feedback. What has been your experiences good or bad within projects?

Dan Linstedt

Posted January 31, 2008 2:47 AM
Permalink | 1 Comment |

1 Comment

Dan -

Great stuff. As I am currently working on a couple of huge projects without these crucial documents (BRD's, TDD's, OBS's, etc. etc.) - this really hits home. It is also truly amazing how these large consulting companies bastardize there own methodologies and skip these essential documents.

Leave a comment

Search this blog
Categories ›
Archives ›
Recent Entries ›