Blog: Dan E. Linstedt Subscribe to this blog's RSS feed!

Dan Linstedt

Bill Inmon has given me this wonderful opportunity to blog on his behalf. I like to cover everything from DW2.0 to integration to data modeling, including ETL/ELT, SOA, Master Data Management, Unstructured Data, DW and BI. Currently I am working on ways to create dynamic data warehouses, push-button architectures, and automated generation of common data models. You can find me at Denver University where I participate on an academic advisory board for Masters Students in I.T. I can't wait to hear from you in the comments of my blog entries. Thank-you, and all the best; Dan Linstedt http://www.COBICC.com, danL@danLinstedt.com

About the author >

Cofounder of Genesee Academy, RapidACE, and BetterDataModel.com, Daniel Linstedt is an internationally known expert in data warehousing, business intelligence, analytics, very large data warehousing (VLDW), OLTP and performance and tuning. He has been the lead technical architect on enterprise-wide data warehouse projects and refinements for many Fortune 500 companies. Linstedt is an instructor of The Data Warehousing Institute and a featured speaker at industry events. He is a Certified DW2.0 Architect. He has worked with companies including: IBM, Informatica, Ipedo, X-Aware, Netezza, Microsoft, Oracle, Silver Creek Systems, and Teradata.  He is trained in SEI / CMMi Level 5, and is the inventor of The Matrix Methodology, and the Data Vault Data modeling architecture. He has built expert training courses, and trained hundreds of industry professionals, and is the voice of Bill Inmons' Blog on http://www.b-eye-network.com/blogs/linstedt/.

On one of my last blogs, I received an interesting comment. I've requested via email, clarification of the term "redundant synonyms", and am hopeful that I will yet receive a reply. However, I wish to expound a little bit on some of the nature of architecture and design, in terms of what I've seen and worked through in the past 13 years in the industry.

In this blog, I will explore deeper, the business meaning - having clear, consistent, and repeatable design architecture. The methods of applying standards to data modeling are also discussed here.

Welcome back to my fire-side chat. As I recline in my rocker, I invite you to sit back, put on your house-shoes and relax a little with me. I always welcome your feedback, and please - feel free to comment, I'm open to learning new things particularly when it's an area of interest.

Down to business... Data Modeling, Information Modeling, Business Modeling - can it, should it, be one in the same? No, probably not. There is certainly a difference between Information and Data, and there is a visible difference between Information and Business.

But what happens when these modeling techniques diverge from business too much? Costs rise, Impacts rise when change is needed, inconsistencies in architecture bubble up, band-aids appear then IT says: Stop, we either need to buy an out of the box solution to replace this mess, or we need to re-write this spaghetti from the ground up. It's as if we started with a single rose bush, and ended up with a thicket - and we don't know where the starting point or ending point is to trim it.

By the same token, they cannot be exactly the same - hence the different modeling methods: business process modeling, data modeling, and systems architecture modeling, and so on. Wait a minute... Where's Information in all of this? Isn't there some standard on Information Modeling? No, there really isn't, probably because it's the grey area that crosses between Data and Business, and it's usefulness to business.

They can however be bound together by similar needs, similar architecture so that when the business needs change, the underlying data model can change without the heavy business impacts, and without the high cost of maintenance, and without severe divergence. But that's a topic for another day. Let's get on to STANDARDS in Data Modeling.

For years, people have insisted on telling me that "no two data warehouses can be modeled the same way." I beg to differ, why then does Universal Data Models enjoy such a large success? It doesn't stop there, but goes on: why then does CRM have competing vendors with very similar feature/functionality sets? (Probably very different models under the covers), they bear the high cost and brunt of changes... No wait, the customer who decides to upgrade undergoes brain surgery every time a major upgrade is released...

Is it because the data warehousing or Integration industry hasn't been bold enough to step forward and proclaim a data model as a standard starting point? But it's more than that, it's not just the data model that's important - it's the architecture and design of the data model that's important. The architecture provides the guidelines for the infrastructure from which the enterprise builds it's vision.

I liken it to this: a two-story house has many many designs, and as long as the foundation and support beams are in the right place, can be created very differently. Compare that to a 32 floor high-rise office in the city. Limited foot print, must rise straight up (usually), must be flexible in high winds at the top, must withstand earth-quakes (in California), glass must be shatterproof (extra weight). and of course the pylons and support structure (infrastructure) must be solid.

I'm no expert in high-rise buildings, but I'm going to assume that the building codes get stricter, the larger the structure that is to be built. Now imagine, after they frame a high-rise and the owner wants to "move" where the elevator shaft is, how possible is this? What if the owner of a two-story house builds an elevator, and wants it moved during framing? Compare the costs, the impacts - much different.

But that's where the similarities end. In the architecture side of the house - there's a ton of planning, design, and standard architecture, proven architecture (reusable, redundant, and consistent set of standards) that govern the build of a high-rise for success. In a two-story house, there's more lee-way, still - standards exist but it can be "thumbed" up to size. Can you take a bunch of two story houses, stack them on top of one-another and make a high-rise out of it? Probably not going to work. Can a high-rise hold a bunch of "two-story" housing units? Yes, if it's partitioned that way.

The bottom line is, it's time to converge our modeling efforts, it's time to provide some consistency, repeatability, and standards to our designs/architecture - whether we're modeling our data, our information or our business - they should all tie together.

I've seen too many data models where the data modeler points to the top left and says: here we implemented X architecture to get around this problem, down here (bottom right) we built Y because the source system had an issue, and over here (top right) Z is in place because it was built before I got here.... and the story goes on. Band-aids to the architecture because the original architecture and data model no longer meets the needs of the business.

It's best to start with a foundational data modeling approach that lends itself to a repeatable and consistent design when it's being extended to meet new enterprise needs. In this manner, metadata can also be captured through naming conventions and architectural design.

More later... Hope this is a useful topic for everyone. Feedback?


Posted April 27, 2005 8:29 PM
Permalink | 4 Comments |

4 Comments

James,
love your comments - read through your blog archive that you posted, and there's a specific reason why I haven't given UML extra time (yet). I was talking to Grady Booch about 2 years ago regarding the Data Vault modeling technique and UML, and he got busy. I'm attempting to re-connect with him at the moment. I expect I will blog on the UML paradigm soon.

Regarding the SOA architecture requiring common/standard data modeling approach, I couldn't agree more, it's just a question of which one.

Regarding visual explanations, I like this site:
http://www.edwardtufte.com/tufte/

And of course Grady's site:
http://www.booch.com/architecture/blog.jsp?archive=2004-05.html

yes tufte and booch are both stupidly smart. although you might want to call alan brown...

This is very interesting and informative, but I got a question:
What do standards contribute to the success of a project?

It might look vague, but I am writing a presentation on this, and I have tried to generalise the issue.
But anymore ideas would be helpful

Sorry, but this link http://blogs.ascential.com/soa/archives/serviceoriented_architecture/index.html

doesn't work. ¿Would you please, send me information about SOA and DW or where I can find Michael Curry article?
Thanks in advance

Leave a comment

    
Search this blog
Categories ›
Archives ›
Recent Entries ›