Originally published October 26, 2010
This article is the second in a series that will address the 12 myths about Agile business intelligence (BI). You can find the complete list on my BeyeNETWORK blog. You can read the full article on Myth 1: Agile BI is Radical and New on my ByeNETWORK Agile BI Channel.
I’m not sure why some folks believe that using Agile BI methods implies that there is no need to produce documentation, and believe me, I’ve heard this many times. Perhaps it stems from the 10th point of the Agile Manifesto, “Simplicity – the art of maximizing the amount of work not done – is essential.” However, my reading of the Agile Manifesto certainly doesn’t cause me to draw that conclusion. I do think, however, that we can take a lesson from “maximizing the amount of work not done” that I’ll discuss later in this article.
Let’s get real! Regardless of the delivery method used, whether it’s waterfall, iterative or Agile, we must produce documentation. Documentation exists in many forms – project charters, project plans, requirements and design specifications, testing documents and production support procedures, to name a few. Documentation serves a very important purpose in that it allows us to communicate and commemorate decisions and outcomes (scope, design, operating procedures, etc.) needed to build, enhance and maintain the solution. Without it, we would not be able to support the delivered solution, and it would be extremely difficult to communicate outside the Agile Scrum team without it. Finally, documentation is needed, if for no other reason, than to stay in compliance with laws and mandates such as the Sarbanes-Oxley Act of 2002, HIPAA (Health Insurance Portability and Accountability Act of 1996), and Basel II.
I’ve also been asked if Agile BI suggests that we should create “light documentation.” Even if I knew what that meant, the answer is still a resounding “No!” The Agile teams that I’ve been involved with have created some very detailed design and test specifications that would make any CMMI level 5 shop jealous. And what’s really cool, the more detailed the specifications, the lower the defect rate. Don’t misunderstand, I still value “working code” over documentation; I just think we need to strike the right balance of documentation to achieve the highest team velocity.
When architecting the Agile BI approach used by my business intelligence services company, ICC, we made sure to include documentation needs in both our estimating methods (project and story point estimating) and in our tracking charts (project performance metrics). As a matter of practice, we create separate story cards to account for documentation. For example, we might create a Business Value Chain story card titled, “Create User Guide” and a Technical Value Chain story card might be titled, “Create Design Specification for Type 2 Dimension.” In doing this, we can regularly allocate hours and/or story points to documentation tasks.
Whether your objective is to provide business intelligence for sales, business intelligence for marketing or business intelligence for CRM, I do believe there are better ways to create and maintain documentation around certain artifacts such as requirements specifications, dimensional database designs, business glossaries and ETL mapping specifications. We’ve historically created these documents in Microsoft Word, Excel and design tools such as ERwin. We regularly hand over 30 to 100 page documents to the business for review and approval, hoping to make sure that it covers everything they could ever ask for. In other words, we bury their needs.
Recent articles by Jim Gallo