We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Business Intelligence Team Structure Phase Alignment

Originally published August 24, 2009

Even under the best of circumstances, it is often very challenging to pull together the right team to deliver a large business intelligence (BI) solution. BI solutions have many components and the projects themselves take on very different personalities during various phases of their development. One of the keys to success in delivering a complex BI solution is to have the right people with the right mix of skills, experience and capabilities at the appropriate times in the engagement life cycle.

Because we are focusing on project teams, we will look at the teams in the context of a stand-alone project following a business intelligence development life cycle (BIDLC) and will not delve into the complexities introduced when looking at a project in the context of a larger program. Also, we will assume we are building a generalized BI solution that requires us to extract data from several source systems, transform and move that data into a data warehouse, and develop a presentation layer for the business users to access the data.

alt

Figure 1:
Reference Solution

While each large BI project is unique in some way, most engagements generally follow a BIDLC. Figure 2 shows a typical iterative BIDLC, including the incremental nature of BI solutions.

alt

Figure 2:
BI Development Life Cycle (3-6 Month Increment)

By following the philosophy that form follows function, we can map the required roles to each of the major phases of the BIDLC. Identifying the roles will help us identify the team structure. What we will find is that the team will shift and evolve throughout the life cycle rather than being a static structure that is assigned up front and is the same from beginning to end. However, we will also find that there are key roles that need to remain constant throughout the engagement in order to get the best results.

To get started, let’s look at a generalized team structure. This structure highlights the two perspectives that BI projects need to consider: business and technology.

alt

Figure 3: BI Team Structure

As we well know, the business drives the solution requirements, and the technology should be designed and built to support the business needs and processes. Therefore, the BIDLC directs us to analyze the situation as the first step.

So, let’s look at what is needed in a project team working on the Analysis Phase. Begin by identifying what needs to be delivered in each phase and then defining what skills are needed to produce that deliverable. From there, we can easily identify what types of people need to be involved at each phase of the project. I am making the assumption that a project management role is required and active throughout the entire life cycle.

In this approach, one role doesn’t necessarily equate to one resource. It is possible that one resource could cover more than one role if that resource possesses the right set of skills. It could also be the case that more than one resource is needed to cover just one role if it is a large project and the amount of work requires it.

The Analysis Phase in a BI solution focuses on identifying the information needs of the user community. The information needs, in turn, drive the functional requirements. The functional requirements are then the basis for the technical and data requirements. Each of these areas is synthesized and presented as a deliverable in this phase. The table below shows these deliverables and the skill set required to fully develop each one.

alt

Table 1: Deliverables and Skill Sets Required

The actual methodology that you use may have somewhat different or more detailed deliverables for this and other phases. However, the deliverables shown here should encompass most or all of what the majority of methodologies recommend. This phase is more business oriented than technically oriented. The resources working on this phase should have good communication skills and a good understanding of business issues and how those issues are impacted by technology.

We can see from the table above that five roles (in addition to the project management role) have been identified:
  1. Business analysis

  2. Functional analysis

  3. Process analysis

  4. Data analysis

  5. Technical architecture
Plugging this information back into our high level team structure, we have the following structure that is appropriate for the Analysis Phase.

alt

Figure 4: Analysis Phase Team Structure

The size and scope of the project will dictate how many resources are needed to satisfy this structure. On a “typical” mid-size project, you might have one project manager, one lead business analyst, one technical architect and one data analyst. The business analysts would likely cover the business analysis, process analysis and functional analysis roles. The people filling this role should also have industry knowledge. The business analyst is responsible for the Information and Functional Requirements deliverables.

The lead technical architect references the Information and Functional Requirements and is responsible for the Technical Requirements document. The data analyst works with the business analyst and is responsible for the Data Requirements. The project manager ensures that the team works together and that the deliverables agree.

The Design Phase will require a slightly different mix of skills and roles. Here the business needs are translated into a technical solution. Based on what they have learned about the business, the team should be able to make the transition to the technical vision of the solution.

The Design Phase will focus on designing the overall solution architecture, data flows, processes, data architecture, data models, ETL architecture and reporting (presentation layer) architecture. Once this information has been compiled, the project manager can refine the project plan for the development phase. The table below shows the deliverables, along with the required skills sets for each deliverable.

alt

Table 2


Depending on your particular business intelligence solution, you may also have a need for customer data integration (CDI), analytics or security, to name a few. The key is to ensure that there is a team member with the appropriate skill set and bandwidth to complete those particular tasks.

While we still have pretty much the same team as in the Analysis Phase, we have moved the data analyst over to the technical side of the project to work more closely with the lead technical (solution) architect. This assumes the data analyst is also a very strong data modeler and can model both the normalized data warehouse and the dimensional data marts (if that is what the solution requires). If not, a separate data modeler should be added to the team at this time.

We also need to bring in an ETL architect, as ETL is typically the biggest and most complex component of a BI solution. For reporting, a package is typically used so the project will require someone with knowledge of the selected tool to ensure that it works well within the given architecture.

The team has now grown to accommodate these additional roles. The project manager, business analyst and technical lead are still playing the major roles on the team, but additional team members will probably be required to cover the additional activities included in this phase.

alt

Figure 5: Design Phase Team Structure


Once the design is completed and approved, the project moves into the Development Phase. This is typically the longest phase of the project and requires the highest number of resources. It is also the most technically oriented of the phases. Most of the team will be focused purely on system development, with the exception of the business analyst and project manager, who will be helping to resolve business issues that emerge. During the development phase, all of the designs are implemented by a team of engineers assigned to the various components of the solution.

There will be several development roles using different technologies during this phase, and those are identified in the table below. In addition to the actual development, there are two key roles that deserve special attention. The first is the solution architect. The person in this role needs to be a very experienced technical resource that has a wide and deep understanding of the solution and each of its components. This role is the glue that helps the whole solution come together and function as planned. The list below outlines the tasks this individual performs and illustrates how vital this role is to large technical projects.

Create the technical design, including:
  • Defining and implementing the "big picture" view of the solution; ensuring that the foundation can be enhanced in the future; providing the vision of how the system will be implemented

  • Defining common processes and procedures

  • Naming and coding standards

  • Creating appropriate use cases, process models, flow diagrams, coding templates, etc. in order to allow ETL programmers to create consistent, high-quality ETL code

  • Defining "operations ready" software, including error handling, reprocessing and controlled exit (close files, cleanup, log messages, etc.)
Perform quality audits, including:
  • Ensuring standards are followed

  • Ensuring common code is used

  • Ensuring good programming practices are followed, including appropriate "refactoring" to ensure maintainable, efficient quality code (e.g., controlled error processing, limited scope of each function, reuse, well commented, etc.)

  • Performance testing and tuning
Provide senior level leadership to the developers:
  • Presenting standard software engineering approaches to common problems

  • Designing creative solutions to new, complex problems

  • Teaching and mentoring more junior programmers
Lead the documentation effort:
  • Ensuring high-quality, accurate, usable operations, programming and use documentation is produced

  • Testing the documentation against the software
Some of these activities will occur during design and others will continue through development and testing. The person in this role will be very busy working with the rest of the team to ensure an integrated, optimized solution.

The other key role is the business analyst. Inevitably, issues will come up during the project that require the business users to make a decision or provide additional information. The business analyst is the liaison between the technical team and the business community. This person helps to resolve issues from a business perspective rather than having the developers guess at what was intended within the requirements and design documents or make a decision based on technical needs without fully considering the business implications.

Here is a table showing the deliverables and roles for the Development Phase:

alt

Table 3: Deliverables and Roles for Development Phase

The team has now expanded to include the rest of the development team. The actual number of team members on the ETL team and reporting team depends on the scope of your project. The data analyst often fills the role of quality assurance and testing. Typically, they are experienced with manipulating data and confirming that information is being accurately moved from the source system, conformed into the data warehouse and ultimately reported correctly in the reporting tool. The team structure for this phase is shown in the chart below.

alt

Figure 6: Development Phase Team Structure

Now that the system has been fully developed and tested, the final phase is Deployment. During the deployment phase, the new system will be fully released to the user community and the development team will turn over the system to the production support staff. For the most part, the resources involved with the heads-down coding are now done. However, some resources knowledgeable about the code are needed to help train the production staff and help track new bugs that are identified in the system. This means that the remaining team once again has to leverage their social and communications skills with both the end users and the production support team. The better the team can convey the required information about the system, the smoother the transition will be.

The primary deliverables and associated roles are shown in the table below:

alt

Table 4: Primary Deliverables and Associated Roles

The team ramps down for this phase and now primarily provides support as the business community begins to use the solution and the production staff takes over the operations. The team will likely be sitting side-by-side with the production team to help them learn the system and will probably go through the first few weeks of bug fixes with them. This will give the production team time to absorb all of the features and functions of the system and build confidence in their ability to support it. This time can also be spent reviewing and enhancing the documentation so that it is as useful as possible for the production staff.

The user training involves showing users the reporting system, getting them familiar with the reporting tool, and helping them to understand the data so that they can use the system as effectively as possible. Ample documentation should be provided to the users so that they can become as self-sufficient as possible.

The chart below shows the team structure as the project winds down during the Deployment Phase.

alt

Figure 7:
Deployment Phase Team Structure


This team should be adequate for the turnover, assuming the technical lead can cover the DBA tasks and the reporting implementer can speak to report development. The business analyst will likely be answering data questions and showing the users how to get the most out of the solution.

Summary

This article has taken you through the staffing cycle of a large project. It demonstrates that a full team is not required throughout the entire project, but there are certain roles that need to be in place at specific times during the project or for the duration of the project. The charts below are two different views of how the staffing levels vary over the course of the project and who can be expected to be on the project when.


Figure 8: Project Personnel by Week (6 Month Iteration Example)

(mouse over image to enlarge)



alt

Figure 9: Staffing Level by Phase

This hypothetical iteration is scheduled to be completed in twenty-eight weeks and has a peak of eleven resources dedicated to the project. I assumed there would be two ETL developers in addition to the ETL lead and two report developers working with the reporting tool implementation resource. Your project will likely vary by duration, scope, or size so your resource levels will vary accordingly.

In conclusion, here are some keys to staffing success at each of the phases mentioned in this article.

Analysis Phase:
  • Use a business analyst who knows the industry as well as the functional area.

  • The technical lead must collaborate closely with the business analyst to ensure synergy between the business needs and the technical solution.

  • Find a data analyst who knows how to examine the data and can understand the information needs of the business users.
Design Phase:
  • Make sure the technical designers understand the business processes, source systems and end users.

  • The technical architect needs to know what tools will be used and how they work.

  • The business analyst must be able to ascertain if the proposed solution will support the business needs.

  • The data analyst must work with real data.
Development Phase:
  • The technical (solution) architect must be able to provide the development standards and frameworks that the entire team will adhere to.

  • Keep the business analyst involved to resolve issues encountered during development.

  • A developer cannot be a substitute for an architect (i.e., a person who has only done development does not have the same skills and experience as an architect).

  • Everyone must document.

  • Programmer/analysts will be more effective and require less supervision than people who are just programmers.

  • The project manager will be needed full time to remove roadblocks and support the development effort.
Deployment Phase:
  • The technical resource(s) that remain must know the solution in minute detail.

  • The end users must participate in training and learn the data.

  • More reports will be requested, so keep a report developer around to help support the first round of requests for new or modified reports (if a report doesn’t work, the end users will see the whole system as broken!).

  • The business analyst will be able to help the end users get the most out of the solution.

  • Make sure the production team is truly ready to take over and support the solution.
One final note: The most important staffing consideration is that the roles are covered and that the people covering those roles have enough time and resources to do their tasks sufficiently. One person can cover more than one role if they have the right mix of skills. The project will be put at risk if a person is assigned to key roles without the proper training or experience.

Recent articles by Todd Saunders

 

Comments

Want to post a comment? Login or become a member today!

Posted January 18, 2010 by Todd Saunders tsaunders@connectknowledge.com

Over the years, I've seen both approaches work and both approaches fail, and both approaches end up somewhere in between.  So many other factors contribute to the overall success of a BI program that I don't believe it's possible to blame success or failure solely on the development methodology.  The metholodogy is a means to an end.  But rigid adherence  to a methodology, especially in the development of a BI solution, can turn into an impediment. 

BI solutions require flexibility, which agile supports.  But BI solutions also benefit from some longer-term planning and foresight, which is supported by SDLC.  The biggest factor to success, besides executive sponsorship, business participation, and adequate budget, comes down to the quality of the team members.  The team determines how it is going to apply the given methodology, how flexible it is going to be with the methodology, and how well the team is going to function within itself.  In summary, a good team can succeed even with a bad methodology.  However, a good methodology cannot fully compensate for a bad team.

Is this comment inappropriate? Click here to flag this comment.

Posted September 3, 2009 by Borsi Evelson bevelson@forrester.com

This is a good recommendation for developing most enterprise applications. Unfortunately, it does not work well for BI. It's been proven time and time again that traditional waterfall SDLC does not appy well for building BI apps. Agile methodology with emphasizes reactions vs. planning, interactins vs. documentation, and rapid prototyping vs. development is what works best.

Is this comment inappropriate? Click here to flag this comment.