We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

The 12 (or so) Myths about Agile Business Intelligence – Myth 4 Agile Business Intelligence Displaces the Software Development Lifecycle in its Entirety

Originally published December 8, 2010

This article is the fourth in a series that will address the 12 myths about Agile Business Intelligence (BI) listed on my BeyeNETWORK blog.

Full articles on the first 3 myths can be found on my Agile BI Channel:

From my dealings with many clients and other business intelligence consultants, I’ve heard some misconceptions about Agile BI and how it’s being viewed from an end-to-end delivery perspective. Specifically, I’ve heard that Agile BI is comprehensive enough to address the totality of traditional software development lifecycles (SDLCs). While Agile BI is comprehensive and rigorous, it does fall short in some of the beginning and ending processes associated with the entire SDLC. However, as you’ll see, there are ways to merge Agile BI with these peripheral activities.

Each organization I’ve worked with has its own SDLC, associated processes, entry/exit criteria and deliverables. While each one was somewhat unique, once you account for nuances, they generally have many things in common that can be generalized into the Larissa Moss and Shaku Atre methodology defined in their book, Business Intelligence Roadmap.

Figure 1: Business Intelligence Roadmap Methodology Overview1

(click on image to enlarge)

For purposes of this article, I’ve added a 17th process step, namely “Operate and Maintain,” to account for post-production support. This will allow me to discuss how problem reports (bugs) and enhancement requests get handled.

Figure 2: Extension to Business Intelligence Roadmap Methodology

(mouseover image to enlarge)

As I’ve explained in the other myth articles, I helped architect my company’s Agile BI methodology used to provide our business intelligence services in a manner that is based on a business context. It focuses on the value chain concept, and when overlaid onto the preceding diagram, we end up with this view that also includes the typical gating processes required by most organizations.

Figure 3: ICC’s Agile BI Framework within the Entire SDLC
(mouseover image to enlarge)

Note that I’ve drawn lines to denote what I believe are the traditional boundaries of Agile BI. Given the above, the question becomes whether or not Agile BI methods can indeed replace the entire SDLC, and if not, what’s not covered? More importantly, can Agile BI methods be merged with these peripheral activities? From my knothole, Agile BI methods cannot replace the entire SDLC, specifically in the following areas:
  • BI/DW strategy justification

  • Initial architecture definition

  • Implementation

  • Operations and maintenance
While Agile BI cannot displace the SDLC in its entirety, Agile BI methods are directly related to these peripheral activities and, in some cases, can be merged with them.

Business Intelligence / Data Warehouse Strategy Justification

As is often the case, the decision of whether or not to pursue a business intelligence / data warehouse (BI/DW) strategy in the first place may or may not be considered to be a part of a SDLC. As this decision seems to be more of a strategy, it appears to fit more into the category of a program, so it either sits outside the SDLC or is a precursor to project execution. This initial justification may include a decision to deploy the solution through a series of related projects that use Agile BI methods, but I really don’t see how Agile BI methods aid in the initial decision-making process.

Initial Architecture Definition

Much like the initial strategy/program justification, architecture decisions are precursors to projects. Given the scope of the BI/DW program defined during initial justification and the macro set of requirements defined by it, the architects craft an architecture and delivery framework as well as create the baseline list of tools and infrastructure needed to support the total solution. Again, these definitions tend to be viewed as onetime strategy/program activities needed to start the first project (but are revisited throughout the life of the program). Therefore, I don’t see how Agile BI methods lend themselves to defining the initial architecture.

As I’ve explained in the Myth 3 article, my company’s approach to Agile BI extends the project backlog in a way that allows the project team to consider the effects the new project will have on the existing architecture. So, while the Agile BI delivery approach is not used to define the initial architecture, it is highly related to initial tasks (justification, infrastructure, planning) associated with projects that exist within the BI/DW program.


After reflecting on the previous paragraphs, I felt a need to address implementation tasks. Business Intelligence Roadmap implementation tasks include:
  • Plan implementation

  • Set up production environment

  • Install all business intelligence application components

  • Set up production schedule

  • Load production databases

  • Prepare for ongoing support2
I do believe that each of these key tasks can be articulated in the form of a story card that focuses on the operational aspects of the project, and I’m thinking that it may be time to either add or modify a value chain to address these key post-development activities. Before doing so, I’d love to hear from you about how the configuration management and operations groups are viewed. Are they generally considered to be a part of a project team or are they peripheral organizations that sit outside the team?

Regardless, there’s no doubt in my mind that the Agile BI team does need to create a set of tasks associated with preparing for implementation. Because I believe that the bifurcation of development and production responsibilities is a best practice, the onus for implementation shifts from the Agile team to the production and support teams and the question then becomes, should the Agile BI framework consider production build tasks? Hmm, a real poser!

Operations and Maintenance (O&M)

Post-production support, aka O&M, historically sits outside the delivery framework, and at initial blush doesn’t lend itself to Agile BI. This is where operational procedures get created, production schedules get managed, help desks earn their keep and production-related issues get identified, escalated and resolved. However, there is a direct link between the support groups and the Agile BI team.

Figure 4: Accounting for Operations and Maintenance within the Agile BI Framework
(mouseover image to enlarge)

As bugs or enhancements get identified, they get placed on the release backlog and should get considered when planning for the next release. The bug fix or enhancement request should therefore be committed to a story card so that the project sponsor can assign it to a future release. When the next planning cycle comes around, the decision to include/exclude the bug/enhancement can be made along with whether to increase the team size to allow for the additional work, or to eliminate some planned content in lieu of the bug fix/enhancement. The project sponsor cannot simply add more work while holding capacity constant (I suppose one could, but this is antithetical to Agile BI methods).

In summary, Agile BI methods focus on the delivery aspects of a SDLC and don’t necessarily apply to initial program activities (justification, architecture definition). They do aid in the definition of implementation and O&M processes and tasks, which may in turn result in new story cards. Agile BI methods must consider early project tasks (justification, architecture, planning) and the outputs of each Sprint must consider, and are directly related to, the business intelligence implementation and support tasks. In my next article I will address the fifth myth – “Agile software development methods can be applied without modification to BI/DW projects” as we head down "The Path to Agility."

  1. Business Intelligence Roadmap, Larissa Moss and Shaku Atre, Chapter 0, p. 15.
  2. Ibid, pp. 352-356
  • Jim GalloJim Gallo
    Jim is  National Director, Vice President, Business Analytics at Information Control Corporation (ICC), a firm focused on reducing the cost of developing BI solutions. Jim and the ICC team have enabled companies to increase the velocity of their development by adopting Agile methods for BI, with a particular focus on turning theory into reality.

    Jim is a recognized expert and has published a number of articles on the practical realities of business intelligence and data warehousing. He is a regular speaker at conferences and industry-related events. He has led a number of large, complex BI projects for Fortune 1000 companies in addition to delivering value to federal and state governments and international clients. For two years in a row, Jim has been named IBM Champion for Information Integration and Federation. He can be reached at jgallo@iccohio.com.

    Editor's Note: More articles and resources are available in Jim's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

Recent articles by Jim Gallo



Want to post a comment? Login or become a member today!

Be the first to comment!