Blog: Wayne Eckerson Subscribe to this blog's RSS feed!

Wayne Eckerson

Welcome to Wayne's World, my blog that illuminates the latest thinking about how to deliver insights from business data and celebrates out-of-the-box thinkers and doers in the business intelligence (BI), performance management and data warehousing (DW) fields. Tune in here if you want to keep abreast of the latest trends, techniques, and technologies in this dynamic industry.

About the author >

Wayne has been a thought leader in the business intelligence field since the early 1990s. He has conducted numerous research studies and is a noted speaker, blogger, and consultant. He is the author of two widely read books: Performance Dashboards: Measuring, Monitoring, and Managing Your Business (2005, 2010) and The Secrets of Analytical Leaders: Insights from Information Insiders (2012).

Wayne is founder and principal consultant at Eckerson Group,a research and consulting company focused on business intelligence, analytics and big data.

October 2013 Archives

The most optimal BI organization adopts a federated structure which blends a mix of centralized and decentralized features. I discussed the basic features of a federated BI organization in my last blog entry and the journey most organizations take to get there. (See "Organizing the BICC Part I: Move to the Middle"). This blog discusses the division of responsibilities between a corporate BI team and embedded BI teams in a federated BI organization.

Organizational federation is not a new or unique concept. For instance, most countries have a federated government. In the United States, federal, state, and municipal governments share civic responsibility. The federal government funds and manages issues of national concern, such as defense, immigration, and interstate commerce, while state and municipal governments fund and oversee local functions, such as policing, transportation, and trash disposal. However, there is a fuzzy boundary between jurisdictions, requiring federal, state, and local officials to collaborate to meet the needs of citizens.

The same is true in the world of BI. Corporate and departmental BI teams focus on different tasks but ultimately share responsibility for the delivery of BI solutions and must work in concert to meet business needs. However, there is a reasonably clear division of responsibility in two areas: data management and report development.

Data Management

The corporate BI team is chiefly responsible for managing data that is shared among two or more business units. In other words, the corporate BI team builds and maintains an enterprise data warehouse (EDW) and dependent data marts geared to individual departments. The corporate BI team also facilitates data governance processes to create standard definitions for commonly used metrics, dimensions, and hierarchies. It publishes these definitions into a data dictionary shared by all business units and embeds them into a business model within the enterprise BI tool.

Report Management

On the reporting side, departments assume the lionshare of responsibility for creating departmental dashboards and scorecards. The BI Reporting Framework (see figure 2) depicts this division of responsibility. The left half of the circle shows the reporting responsibilities of the corporate BI team and right half shows the reporting responsibilities of business units. These responsibilities are divided by the other axis, which represents "top down BI" and "bottom up" BI.

Figure 2. BI Reporting Framework
BI CoE Divison of Responsibility - No Arrows.jpg

Top Down and Bottom Up. In top-down BI, the corporate BI team delivers standard reports and dashboards to casual users. The team gathers requirements, models and sources the data, and then loads it into the data warehouse. Developers then build reports and dashboards that dynamically query data warehouse data. In contrast, in bottom-up BI, power users query the data warehouse directly, in an ad hoc fashion. They explore, analyze, and integrate data from various systems and then create reports based on their findings, which they often publish to departmental colleagues and executives. Basically, top-down BI delivers standard reports to casual users, and bottom-up BI world enables power users to explore and analyze data and create ad hoc reports.

Enterprise BI. Thus, the upper left quadrant in figure 2 represents the intersection of enterprise BI and top-down BI. This is where corporate BI developers (i.e. enterprise) create production reports and dashboards (i.e. top-down) that would be difficult for any single division to create on their own. In the bottom left quadrant, corporate statisticians or data scientists (i.e. enterprise) who are aligned with individual divisions but not collocated, explore and query data in an ad hoc fashion to create predictive learning models.

Divisional BI. In the bottom-right quadrant, business unit analysts in each division use self-service BI tools to analyze data and create ad hoc reports to display their insights. If executives and managers want to continue seeing these reports on a regular basis, the business unit analysts turn them over to the collocated BI professionals who convert them into production reports (top-right quadrant.) This handoff between analysts and embedded BI staff is critically important, but rarely happens in more organizations. Too often, business unit analysts publish reports and then end up perpetually maintaining them. This is a job they aren't trained or paid to do and keeps them from spending time on more value-added tasks, like analyzing the business.

To succeed, there needs to be a bidirectional flow of information between each of the sectors as depicted in figure 2.

Figure 2. Handoffs between BI Sectors
Handoffs.jpg

Going around the diagram, we can describe the handoffs that occur between each group in a BI Center of Excellence:

  • Enterprise BI Team-->Departmental BI Team (Top left to top right): The enterprise BI team has data and reporting professionals who specialize in specific tools. Besides creating complex, cross-functional reports and dashboards, they provide first line-of-support to embedded BI professionals who are more business oriented than tools oriented.
  • Departmental BI Team-->Enterprise BI Team (top left to top right): Conversely, the embedded BI professionals help gather requirements for complex cross-functional applications and communicate them to enterprise BI reporting specialists.
  • Departmental BI Team-->Departmental Analysts (top right to bottom right): Embedded BI professionals provide first line of support to business analysts who need to learn how to use self-service BI tools to explore data and create ad hoc reports.
  • Departmental Analysts-->Departmental BI Team (bottom right to top right): Conversely, analysts hand over their ad hoc reports to embedded BI professionals to convert them into production reports.
  • Departmental Analysts-->Data Scientists (Bottom right to bottom left): Departmental analysts submit requests to data scientists for more complex analyses than they can perform and work with them to gather requirements and data sets.
  • Data Scientists-->Departmental Analysts (Bottom left to bottom right): Conversely, data scientists aligned with a department provide insights to business analysts based on data models they've developed.
  • Data Scientists-->Enterprise BI Team. (Bottom left to top left): Data scientists deliver model scores to the enterprise BI team, which incorporates them into complex reports. (Ditto with departmental reports.)
  • Enterprise BI Team-->Data Scientists (Top left to bottom left): The enterprise BI team delivers data and requirements to the data scientists who use them to create analytical models.
Bridging the disparate worlds of top-down and bottom-up BI and enterprise and divisional BI is not easy. It requires a lot of communication and collaboration. The structure that glues together these BI sectors requires both matrixed reporting relationships and a multi-layered governance body. This is the topic of my next blog.

Posted October 17, 2013 10:54 AM
Permalink | No Comments |

The origin of every business intelligence (BI) team is quite simple. A company starts doing business and then recognizes it needs to track how it's doing. So business unit heads hire analysts who create a bevy of reports using whatever low-cost tools they can find, usually Excel and Access. Although managers get answers to their questions, it comes at a high cost: analysts spend more time integrating data than analyzing it and create a bevy of redundant data silos that are costly to maintain and prevent executives from getting a single, integrated view of the business. When the CEO can't get an answer to a simple question, such as "How many customers do we have?" or the CFO sees a red balance sheet caused by a proliferation of redundant people and systems, they take action.

Usually, executives decide to move reporting, analysis and data management out of the business units and into a shared corporate service. By centralizing the BI function, executives pull BI-related professionals out of the business units and put them onto an enterprise BI team that is charged with building an enterprise data warehouse and creating all reports and dashboards for business units. The goal of the new group is to align the business with uniform data and deliver cost savings through economies of scale. This reorganization swings the pendulum from a decentralized approach to managing BI to a centralized one.

Problems

All goes well until business units start to grumble that the new corporate BI team isn't meeting their information needs in a timely manner. Whereas in the past, a business person could initiate a project by talking with an analyst in the next cubicle, now she needs to submit a proposal to the corporate BI team, participate in a scoping process, and then wait for the BI team to decide whether or not to do the project. And this is all before a single line of code gets written. Once a project finally starts, the BI team takes longer to build the solution because it's now a step removed from the business unit and doesn't know its processes, data or people well enough to work efficiently or effectively. Given these circumstances, some business unit heads decide they can't wait for corporate BI to meet their needs and hire analysts to build reports for their group, effectively replacing the ones that the corporate BI team "stole" from them.

Finding the Middle

By swinging from one end of the organizational spectrum to the other--from decentralized to centralized--the BI program becomes a bottleneck for getting things done. At this point, some executives throw up their hands and slash the BI budget or outsource it. Yet enlightened executives seek counsel to find a more nuanced way to structure the BI program so that it marries the best of both worlds. In essence, they find a middle ground that I call this a federated BI organizationthat delivers both the nimbleness and agility of a decentralized approach and the standards and consistency of a centralized approach. (See figure below.)

Figure 1. Evolution of BI Organizations
Evolution of BI Teams.jpg

A federated BI team maintains a unified BI architecture that delivers consistent, accurate, timely, and relevant data that makes the CEO happy. And it delivers economies of scale that makes the CFO happy. Moreover, because a federated BI organization embeds BI professionals in the divisions, it delivers BI solutions quickly, which makes business unit heads happy. Finally, through cross-training and support provided by collocated BI professionals, business analysts finally become proficient with self-service BI tools, which makes them happy.

So there is a lot to like with a federated BI organization, and very little to dislike. In essence, this approach creates a common charter that impels the business and IT to collaborate at a deep and more productive level. The only real challenge is managing the web of matrixed relationships and co-managed teams that maintain the proper balance between business and IT. We'll discuss these relationships in a future blog post.


Posted October 17, 2013 10:39 AM
Permalink | No Comments |