Analytical-Mind

We created an Agile Business Intelligence group on LinkedIn a few months ago for people who share interest in Agile and Business Intelligence.

Join us and share your thoughts and best practices.

I wanted to summarize Tuesday morning’s session at the TDWI BI Executive Summit.

Larissa Moss (President of Method Focus) presented “Agile Development: What is it and will it work for BI?” where she explained that after failing at a DW project she was working on using the waterfall approach, she stumbled into Agile in the mid ’90s. After explaining what Agile actually is and how it differs from a traditional waterfall approach, she asked the “$64,000 question“, “Can Agile work for BI?” to which she answered “It depends what you call BI“.

Larissa argues that Agile BI works well for companies that don’t have to deal with the complexity of the data (i.e. don’t deal with building an underlying data warehouse). As such, she explained that using an Agile BI approach to do a data warehouse project where the development team has to deal with end-to-end solution including data standardization, data modeling and ETL cannot work due to the complexity and the un-known associated with the underlying data.

Larissa finished her presentation by proposing a solution to address this situation – Extreme Programming^TM which is a 16 steps Agile methodology she developed to address the pitfalls of the traditional waterfall approach. Her book called Extreme Programming^TM is scheduled to be released shortly.

Jim Hill (Director of Data Management at 1-800-contacts) then presented “Applying Agile Techniques to BI” and how his organization has been successful for the past 4 years in using Agile Development “for all software development projects and BI activities“.

Jim explained the process used by his business users to log their requests (user stories) which are then prioritized weekly and included in weekly iterations. He gave an example of 1-800-contacts’ Call Center BI which is deployed to over 400 agents’ desktops and for which the data is refreshed every 15 minutes.

Following Jim was Wyatt Weeks (Group Manager Business Intelligence at Sport Authority) who presented “Agile Data Warehousing at Sports Authority”. Wyatt told the audience that his team had chosen to use Agile Development for the first time on a “Large, high risk project with short time lines” and following their success, the team has continued to use Agile to deal with changing priorities.

Wyatt’s Scrum team consists of 7 to 9 members and all phases of their BI projects (architecturee, data modeling, ETL, and report development) are delivered through monthly sprints. He highlighted the benefits of Scrum: team collaboration, enforces communication and visibility, and clear definition of priorities. 

The 3 speakers gave good presentations and agreed that indeed Agile Development can be used for BI projects as long as it is adapted to the reality and constraints of the organization.

Timo Elliot asked an interesting question today. In his post, What Might Go Wrong in Business Intelligence in 2009? He points to a few interesting risks facing Business intelligence projects (People will try to do without BI, People will revert to hand-coding and excel macros, Organizations will implement standards, and Business units will find it easier than ever to implement their own solutions).

His post got me thinking. What risks are we going to face as we launch a new BI consulting practice in the current context? I would argue the following:

• For companies who have already launched a BI project, they may want to accelerate pace in order to get benefits sooner and use the intelligence to develop a solid competitive advantage. This is where the advantage of an Agile approach become interesting.
• For companies who were thinking of starting a BI initiative, they must be educated with regards to the benefits of a BI project. In the current context, nobody will want to initiate a 3 years and $2M project. Once again the Agile approach makes sense.
• Companies who are currently using excel macros may not be as bad since they understand the value of business intelligence but lack the right processes and tools. I would argue that those companies need to be presented with a cost-effective tool and process to obtain better insight. They need to be shown that the current process is not sustainable and error-prone.
• The forth risk needs to be addressed differently as the challenge is one of integration. It would be necessary to determine the quality of the current process and resulting information in order to propose once again a more sustainable and centralized solution.

Although it might sound simplistic, it seems to me that the current economic situation simply raises the bar on consulting firms and technology providers. We now have to clearly demonstrate our solutions are cost-effective and will add value in a shorter time frame. Very few people would debate the “added-value” part but the “cost-effective” and “shorter time frame” might be more of a challenge. This is an opportunity for the best companies to stand out of the crowds.

Change always brings opportunities for those who want to grow.

In the context of launching a new consulting practice, I’m reviewing documentation and compiling examples in order to explain to potential customers the benefits of the Agile approach in the context of Business Intelligence.

I’m currently reading Agile Project Management with Scrum and after watching the hockey game between Montreal and Tampa Bay, an analogy came to mind. Can we compare the Scrum approach to project management with hockey?

Although the analogy used by Schwaber in his book is very easy to understand (the Scrum master is similar to a sheppard dog who needs to protect his sheep), I found his comparison to be simplistic and slightly degrading for the development team (sheep?!).

The analogy could have been with baseball but the Expos left Montreal in 2004, or between Scrum and football but are the Alouettes/Concordes/Stallions/Alouettes really that popular? or between Scrum and soccer (the European football) but I preferred to use hockey – being our national sport.

• A hockey game is divided in 3 periods (sprints).
• The coach acts as the Product owner who provides requirements in order to reach an objective – winning the game.
• The team captain is the Scrum master: he/she is a player just like the other players on the team but has additional responsibilities in the context of the game.
• The coach providing feedback between each period is a well entrenched feedback process.

So my question is, does this analogy work for you? Would potential customers be able to easily understand Scrum if you used the hockey analogy?