We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Blog: Ronald Damhof Subscribe to this blog's RSS feed!

Ronald Damhof

I have been a BI/DW practitioner for more than 15 years. In the last few years, I have become increasingly annoyed - even frustrated - by the lack of (scientific) rigor in the field of data warehousing and business intelligence. It is not uncommon for the knowledge worker to be disillusioned by the promise of business intelligence and data warehousing because vendors and consulting organizations create their "own" frameworks, definitions, super-duper tools etc.

What the field needs is more connectedness (grounding and objectivity) to the scientific community. The scientific community needs to realize the importance of increasing their level of relevance to the practice of technology.

For the next few years, I have decided to attempt to build a solid bridge between science and technology practitioners. As a dissertation student at the University of Groningen in the Netherlands, I hope to discover ways to accomplish this. With this blog I hope to share some of the things I learn in my search and begin discussions on this topic within the international community.

Your feedback is important to me. Please let me know what you think. My email address is Ronald.damhof@prudenza.nl.

About the author >

Ronald Damhof is an information management practitioner with more than 15 years of international experience in the field.

His areas of focus include:

  1. Data management, including data quality, data governance and data warehousing;
  2. Enterprise architectural principles;
  3. Exploiting data to its maximum potential for decision support.
Ronald is an Information Quality Certified Professional (International Association for Information and Data Quality one of the first 20 to pass this prestigious exam), Certified Data Vault Grandmaster (only person in the world to have this level of certification), and a Certified Scrum Master. He is a strong advocate of agile and lean principles and practices (e.g., Scrum). You can reach him at +31 6 269 671 84, through his website at http://www.prudenza.nl/ or via email at ronald.damhof@prudenza.nl.

I have written numerous times about efficient and effective ways of deploying data in organizations. How to architect, design, execute, govern and manage data in an organization is hard and requires discipline, stamina and courage by all those involved. The challenge increases exponentially when scale and/or complexity increases.

On average a lot of people are involved and something of a common understanding is vital and unfortunately often lacking. As a consultant it is a huge challenge getting management and execution both on the same page, respecting the differences in responsibilities and the level of expertise regarding the field we are operating in.

An abstraction functions as a means of communication across the organization. An abstraction to which the majority can relate to and that can be used to manage, architect, design, execute, govern and manage the field that is at stake.

For data deployment I came up with the so-called '4 Quadrant model' (4QM).

It starts with a basic assumption that data deployment starts with raw materials and ends up in some sort of product. And in the process of getting raw materials to end products, logistics and manufacturing is required. 

It starts with the basic assumption that reliability and flexibility are both wanted, but are mutual exclusive.

It starts with the basic notion of the 'push pull point', which stems from the logistic- and manufacturing literature we were taught in high school about push systems and pull systems.

PushpullIt starts with the basic notion that that data needs to be separated into facts and context.

Q1q2_v2
There is no single version of truth

There is only a single version of facts

There are many truths out there

Push systems are to be standardized as much as possible. Only if we standardize we can automate. Holding off any product-specific demand features is vital within push systems. However, the table turns completely when we enter the pull-systems domain.

The push pull point is - in architectural terms - a decoupling point, separating fundamentally different concerns, but it is not the only one. Using this high level of abstraction another decoupling is crucial; development style. Missing this one often results in pain, misery, demotivation, failed projects, dissatisfied users, budget overruns, IT focus, etc...

DevelopmentStyle


This separation between systematic and opportunistic development style respects both the professionalism of the data scientists and the IT engineers. It respects both the organizational ambitions as the more local ambitions. 

Two decouple points; push pull and development style. Depicting them results in four quadrants.

4qm

Quadrant IV is especially worth some attention. Typical use cases for this quadrant:

1)   Reducing uncertainty/getting requirements: 'I do not know what I need, what I want', 'I do not know the quality of the data yet'. Experiment a lot, change fast, instant feedback.
Quadrant IV is especially worth some attention. 

2)   Reducing lead times:  Going from quadrant I to II might take to long. There needs to be an environment where products can be made a.s.a.p.

3)   Stimulating innovation: Discover, experiment, throw away, testing hypothesis, etc.. An environment that is as friendly as possible for the creative mind.

In quadrant IV we are architecting some degree of chaos...which is fine.

Finally, it needs to be understood that there is no one way the data flows through the quadrants. Four process variants are depicted in the small picture on the right, but more can be thought of. For example, the quadrant can be used to architect a phased/managed approach towards migration of legacy data deployment systems (which I know you have!!)
 ProcessVariants

The point is that there are several ways of manufacturing information products and you might wanna consider them all. Much to often I only see 1 way (or the highway) -  USSR-Politburo-Fundamentalism - kind of deployment option. 

I can write a book about the four quadrants and the operationalization of the decouple points. But this is a blog post and I really need to stop now.

My point is that the four quadrants are different in almost every way in terms of architecture, design, execution, governance and (project) management:

  • qI and qII need a more centralized governance style, qIII and qIV require a more decentralized governance style
  • qII might be more suitable for agile deployments, while qI might be more suitable for good old waterfall;
  • In terms of data modeling, q1 needs a more temporal style, q2 might be more prone to dimensional;
  • Both qI and qII need to be aligned with Enterprise Data Models, q3 and q4 do not;
  • In terms of tooling, q1 needs more discover/analytic functionality, while q2 is more prone to reporting, dashboards, etc..;
  • Different kinds of databases might be required for the quadrants and even within the quadrants;
  • Ownership of definitions is in q1 more or less driven by source, while ownership of definitions in q2 is more or less driven by the product owner;
  • qII publishes information products which are certified, trusted, centrally maintained. qIV publishes information products where the creator bares all responsibilities regarding trustworthiness, change management, etc..
  • q1 the datamodel and the data logistics need to be standardized and automated as much as possible, there is more freedom (based on the requirements) in q2. In q4 is completely free how products are modeled.
  • q1 might be the domain of the IT department or even prone to be outsourced eventually. Other domains will probably be situated close to the requirements in an organization;
  • q3 is a suitable quadrant to situate sources which are hugely ad-hoc or use new innovative technologies;
  • Every quadrant requires a specific education and competencies profile;
  • etc..

      



    Posted August 16, 2013 5:53 AM
    Permalink | 2 Comments |

    2 Comments

    It would be great if there was a second quadrant which described the charactaristics of the infrastructure/system.

    Maybe such a quadrant could be used to find out questions like:
    - What kind of infrastructure would be 'fit for purpose' for the quadrants?
    - What type of software propagation would be wise?
    - What would be the required focus with regards to the quality of the (different) infrastructures?

    Gr. Roy

    This Quadrant data model is applied in many aspects from design, construction to project management. Posts are really very useful.

    Leave a comment