We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Business Intelligence Architecture: The Dividing Line Conclusion of a 3-Part Series

Originally published September 22, 2011

"If it can't be my design, tell me where do we draw the line." - Poets of the Fall

In Part 2 of this series, we discussed the impact of organizational models on business intelligence (BI) development. In particular, we described conglomerate, cooperative, and centralized models and showed how two companies (Harley-Davidson and Dell) migrated across this spectrum and its impact on their BI architectures and organizations.

As any veteran BI professional knows, business intelligence architectures come in all shapes and sizes. There is no one way to build a data warehouse and BI environment. Many organizations try various approaches until they find one that works, and then they evolve that architecture to meet new business demands. Along the way, each BI team needs to allocate responsibility for various parts of a BI architecture between corporate and business unit teams. Finding the right place to draw the proverbial architectural line is challenging.

The Business Intelligence Stack

Figure 1 shows a typical BI stack with master data flowing into a data warehouse along with source data via an ETL tool. Data architects create business rules that are manifested in a logical model for departmental marts and business objects within a BI tool. BI developers who write code and assemblers who stitch together predefined information objects also create reports and dashboards for business users. Typically, most databases and servers that power operational and analytical systems run in a corporate data center.

Using this high-level architectural model, we can study the impact of the three organizational models described in Part 2 of this article on BI architectures.

Figure 1: Typical BI Stack

Conglomerate – Shared Data Center

In a conglomerate model, business units have almost complete autonomy to design and manage their own operations. Consequently, business units also typically own the entire BI stack, including the data sources, which are operational systems unique to the business unit. Business units populate their own data warehouses and marts using their own ETL tools and business rules. They purchase their own BI tools, hire their own BI developers and develop their own reports. The only thing that corporate manages is a data center that houses business unit machines and delivers economies of scale in data processing (Figure 2).

Figure 2: Conglomerate Business Intelligence

Cooperative Business intelligence – Virtual Enterprise

In a cooperative model where business units sell similar – but distinct –  products, business units must work synergistically to optimize sales across an overlapping customer base. Here, there is a range of potential BI architectures based on an organization’s starting point. In the virtual enterprise model, an organization starts to move from a conglomerate business model to a more centralized model to develop an integrated view of customers for cross-selling and up-selling purposes, and to maintain a single face to customers who purchase products from multiple business units.

In a virtual enterprise, business units still control their own operational systems, data warehouses, data marts, BI tools and employ their own BI staff. Corporate hasn’t yet delivered enterprise ERP applications but is thinking about it. Its first step towards centralization is to identify and match mutual customers shared by its business units. To do this, the nascent corporate BI team develops a master data management (MDM) system, which generates a standard record or ID for each customer that business units can use in sales and service applications (Figure 3).

Figure 3: Cooperative Business Intelligence - Virtual Enterprise

The corporate group also creates a fledgling enterprise data warehouse to deliver an enterprise view of customers, products and processes common across all business units. This enterprise data warehouse is really a data mart of distributed data warehouses. That is, it sources data from the business unit data warehouses, not directly from operational systems. This can be a persistent data store populated with an ETL or virtual views populated on the fly using data virtualization software. A persistent store is ideal for non-volatile data (i.e., dimensions that don’t change much) or when enterprise views require complex aggregations, transformations, multi-table joins or large volumes of data. A virtual data store is ideal for delivering enterprise views quickly at low cost, as well as for building prototypes and short-lived applications. 


Cooperative Business Intelligence – Shared BI Platform

The next step along the spectrum is a shared BI platform. Here, corporate expands its appetite for data processing. It replaces business unit operational applications with enterprise resource planning (ERP) applications (e.g., finance, human resources, sales, service, marketing, manufacturing, etc.) to create a more uniform operating environment. It also fleshes out its BI environment, creating a bonafide enterprise data warehouse that pulls data directly from various source systems – including the new ERP applications – instead of departmental data warehouses. This reduces redundant extracts and ensures greater information consistency (Figure 4).

Figure 4: Cooperative Business Intelligence - Shared Platform

Meanwhile, business units still generate localized data and require custom views of information. While they may still have budgets and licenses to run their own BI environments, they increasingly recognize that they can save time and money by leveraging the corporate BI platform. To meet them halfway, the corporate BI team forms a BI Center of Excellence and teaches the business units how to use the corporate ETL tool to create virtual data marts inside the enterprise data warehouse. The business units can upload local data to these virtual marts, giving them both enterprise and local views of data without having to design, build and maintain their own data management systems.

In addition, the corporate BI team builds a universal semantic layer of shared data objects (i.e., a semantic layer) that it makes available within the corporate BI tool. Although business units may still use their own BI tools, they increasingly recognize the value of building new reports with the corporate BI tools because they provide access to the standard business objects and definitions that they are required or highly encouraged to use. The business units still hire and manage their own BI developers and assemblers, but they now have dotted-line responsibility to the corporate BI team and are part of the BI Center of Excellence. This is the architectural approach used by Intuit (see Part 1 of this series).

Centralized Business Intelligence – Shared Service

Some organizations, like Dell, don’t stop with a shared platform model; they continue to centralize BI operations to improve information integrity and consistency and squeeze all redundancies and costs out of the BI pipeline. Here, the corporate BI team manages the entire BI stack and creates tailored reports for each business unit based on requirements. For example, Dell’s EBI 2.0 program (see Part 2 of this series) reduced the number of report developers in half and reassigned them to centralized reporting teams under the direction of a BI Competency Center where they develop custom reports for specific business units. The challenge here, as Harley-Davidson discovered, is to keep the corporate BI bureaucracy from getting too large and lumbering and ensure it remains responsive to business unit requirements. This is a tall task, especially in a fast-moving company whose business model, products and customers change rapidly.

Summary

Federation is the most pervasive BI architectural model, largely because most organizations cycle between centralized and decentralized organizational models. A BI architecture, by default, needs to mirror organizational structures to work effectively. Contrary to popular opinion, a BI architecture is a dynamic environment, not a blueprint written in stone. BI managers must define an architecture based on prevailing corporate strategies and then be ready to deviate from the plan when the business changes due to an unanticipated circumstance such as a merger, acquisition or new CEO.

Federation also does the best job of balancing the dual need for enterprise standards and local control. It provides enough uniform data and systems to keep the BI environment from splintering into a thousand pieces, preserving an enterprise view critical to top executives. But it also gives business units enough autonomy to deploy applications they need without delay or IT intervention. Along the way, it minimizes BI overhead and redundancy, saving costs through economies of scale.

  • Wayne EckersonWayne Eckerson
    Wayne has been a thought leader in the business intelligence field since the early 1990s. He has conducted numerous research studies and is a noted speaker, blogger, and consultant. He is the author of two widely read books: Performance Dashboards: Measuring, Monitoring, and Managing Your Business (2005, 2010) and The Secrets of Analytical Leaders: Insights from Information Insiders (2012).

    Wayne is founder and principal consultant at Eckerson Group, a research and consulting company focused on business intelligence, analytics and big data.

Recent articles by Wayne Eckerson

 

Comments

Want to post a comment? Login or become a member today!

Posted September 28, 2011 by fritter@clarkstonconsulting.com

Excellent series tying architecture to organizational structures and what that meands from a BI pov. Enjoyed the read.

Is this comment inappropriate? Click here to flag this comment.