Although the notion of Agile Business Intelligence is still fairly recent, I decided to do some research (aka Googled) to find out who are the Top 10 People in the Agile Business Intelligence / Agile Date Warehousing World. Assuming that Google’s ranking is accurate, I compiled the following list of thought leaders.

My objective was to identify the Top 10 and much to my surprise, I was only able to identify 4 people (companies were excluded)! These 4 individuals’ name consistently appeared in the search results.

1. Scott W. Ambler
2. Ted Malone
3. Ralph Hughes
4. Ken Collier

Do you feel there are names that aren’t in the list and should be there? Let me know.

You might be interested in these related posts:

1. Book Review: Agile Data Warehousing
2. Pyxis is currently recruiting for a Consultant – Agile Business Intelligence Developer
3. Agile Business Intelligence – Collaborative Book

Join the collaborative project to write a book on Agile Business Intelligence.

After dreaming about it, talking about it, and researching for it, I am officially launching the “Agile Business Intelligence – Collaborative Book” project.

My objectives are to:

• leverage the wisdom of the crowd in order to write an entertaining book full of insights;
• demonstrate by providing real life examples that Agile principles can be applied to Business Intelligence and Data Warehouse projects;
• collaborate with people from all over the world on an exciting project;
• learn from other people’s experience and
• meet people and have fun in the process.

The book will present the tale of a fictitious organization that successfully develops and implements a Business Intelligence project using Agile principles.

 A link to the draft version of the book will be provided shortly.

I am looking for people who are willing and interested in sharing their experience and contribute to the advancement of Agile Business Intelligence through a well documented and interesting book. Once completed, this book will be published and all revenues from the sale of the book will be donated to non-profit organizations.

In order to become a contributor, please answer the questions presented on the registration form.

You might be interested in these related posts:

1. Agile BI Collaborative Book – Preliminary Chapter 1
2. Agile Business Intelligence Book – On Hold Indefinitely
3. 4e Salon Business Intelligence in Montreal

The presentation given yesterday during the Agile BI – Happy Hour is now available on slideshare. The presentation is currently only available in French but it should be translated shortly.

You might be interested in these related posts:

1. Introduction to Scrum – Shareable Power Point Presentation
2. From Anarchy to Sustainable Development an Experience report
3. Share best practices for an Agile BI project

I'm facing a big problem. Can you help me? *

There are questions and challenges that keep coming up. No matter if you work on the business side or the technical side of your organization, you have certainly faced or might be dealing with some of these challenges and you aren’t quite sure what to do. If the statements below sound familiar to you, you may want to read on.

• My project team constantly misses deadlines.
• The project team keeps exceeding its budget.
• My project team doesn’t deliver on the requirements.
• The end users don’t know what they want.
• The requirements keep changing and that constantly impact our project plan.
• The project team develops software components that don’t seem to have any business value and they seem to produce more paper tha software.   
• My team develops software that doesn’t satisfy my users.
• The project team usually finds problems very late in the development process.
• The project team does not have the right skills.
• The project team is tired, nobody is having fun and we are losing good people.
• I need to wait a long time before the project team gets me the information I need.
• We know we have issues but we don’t know where to start.
• We need to outsource our software development activities in order to cut costs.
• The project team delivers poor quality software.
• We have started using Agile for a small project and our management team wants us to scale it to the rest of the organization.

Agile principles can work for your Data Warehouse / Business Intelligence project but it is critical to determine which business problem you are trying to solve. Below, we are presenting a list of issues frequently encountered and we offer potential solutions to each of them.

What is your challenge?

My project team constantly misses deadlines.

How Agile can help

Implementing Scrum as your project management approach and the proper reporting tools (such as burn down charts) would help you anticipate potential delays and address your delays in a timely manner.

In addition, using frequent face-to-face communication instead of communicating through a project plan will increase your team’s productivity, performance and compliance to defined time lines.

What is your challenge?

The project team keeps exceeding its budget.

How Agile can help

The situation might be worse than you think because you may also be delivering the wrong functionality to your users.

Using long range planning often results in high variance in the actual budget being spent whereas delivering software in short iterations (up to 4 weeks) will allow you to better monitor your costs while ensuring you deliver the expected results.

What is your challenge?

My project team doesn’t deliver on the requirements.

How Agile can help

This could be two different issues – the requirements aren’t well defined by the end users or the requirements aren’t clearly understood by the development team.

In either case, enforcing close collaboration between the end users and the development team by co-locating the team and enforcing face-to-face communication will greatly increase the chances of delivering the right software.

You could turn the situation around by implementing the right processes which would allow you to welcome changes during the development phase and adapt to your changing business challenges.

What is your challenge?

The end users don’t know what they want.

How Agile can help

This is not unusual but it is not necessarily an issue with your end users. With most projects using a traditional software development methodology, end users are asked at the beginning of the project for their requirements and will eventually see the results months or years later once the software has been delivered.

With an Agile approach, end users are asked to define their short term requirements (for the next 4 weeks) instead of defining the entire scope of the project which will help the project team to deliver on the requirements.

What is your challenge?

The requirements keep changing and that constantly impact our project plan.

How Agile can help

Indeed, if you are using a traditional software development methodology you are likely to run into changes as the market dynamics evolve frequently. With an Agile approach, your team will not only learn to deal with changes in requirements but will even learn to embrace those changes and adapt accordingly.

What is your challenge?

The project team develops software components that don’t seem to have any business value and they seem to produce more paper than software.   

How Agile can help

Why are you? The focus of your development team should be to deliver value – working software. If for some reasons, a lot of your development team’s time, efforts and energy are spent working on documents (project plans, requirements, architecture, models, etc.) instead of working on software you probably need to reconsider the software development approach you are using.

Using a pragmatic and realistic approach like agile for your software development process will address the most critical problems typically encountered by a software development team and will greatly increase your return on investment.

What is your challenge?

My team develops software that doesn’t satisfy my users. Satisfying the requirements of your users is critical for a software development team and applying agile practices will greatly help.

How Agile can help

The integration of the business users as part of the development team is a great way to user their requirements are properly addressed. In addition, using techniques such as user stories that describe the features from an end-user perspective makes it easier for the development team to meet the requirements once it is clearly documented.

What is your challenge?

The project team usually finds problems very late in the development process.

How Agile can help

It is frequently recognized that the later you find problems in the process, the more expensive it becomes to fix them. Based on that conclusion, the implementation of an Agile software development approach forces the team to present the outcome of their labor to their users in order to demonstrate and test the software frequently which bring to the surface the issue.

What is your challenge?

The project team does not have the right skills.

How Agile can help

Although it may be possible that your team lacks some skills, it is unlikely that your people aren’t qualified for your project.

Your team might need some help and coaching around some specific software engineering practices and with the right coaching, they can improve their development skills.

What is your challenge?

The project team is tired, nobody is having fun and we are losing good people.

How Agile can help

It has frequently been proven that motivated employees deliver better results so if you want your project and your team to succeed,creating an environment where people can learn, work on challenging project and feel they are contributing to the success of the organisation will greatly help.

What is your challenge?

I need to wait a long time before the project team gets me the information I need.

How Agile can help

The business user and the development team may not be working closely together and you are probably using a traditional waterfall approach where the development team asks for requirements, goes off to develop and test the queries and reports and once they feel comfortable with the results will present them to the business users.

By including the business users within the project team and using an incremental development cycle, you would see the progress being made on your original request and could influence the amount of details required. Keeping small iterations and short feedback loops would allow you to quickly see how the development team is doing and quickly get access to the information you need.

What is your challenge?

We know we have issues but we don’t know where to start.

How Agile can help

An agile approach helps improve communications within the development team and with the business users. Having frequent demonstration of the system’s capabilities will quickly bring to the surface the issues and will allow the team to address them.

What is your challenge?

We need to outsource our software development activities in order to cut costs.

How Agile can help

Not necessarily, a better approach might help you reduce your costs. The Standish Group Study (Reported at XP2002 by Jim Johnson) showed that as much as 64% of system functionality was never used. Using an Agile approach to build the right software will immediately reduce the overall cost of developing software within your organization.

What is your challenge?

The project team delivers poor quality software.

How Agile can help

You may want to consider implementing test-driven development (TDD) where developers need to start by writing their test before developing new code or you may want to leverage pair programming.

In addition to changing the way you develop software, it would be beneficial to use short development cycle and demo the resulting products frequently in order to gain visibility to the software and address the issues in a timely fashion.

What is your challenge?

We have started using Agile for a small project and our management team wants us to scale it to the rest of the organization.

How Agile can help

Absolutely. Now that you have seen first hand the tremendous benefits of using Agile practices, you may want to coaching and guidance to help you roll out the new approach to the rest of your development organization and take full benefits at a larger scale.

* Picture by Mikey G Ottawa used under the Creative Commons (CC) agreement. The view expressed in the blog post is the one of the author. The photographer does not endorse in any way the content of this blog post or the work of the author.

You might be interested in these related posts:

1. Other interesting blog posts (June 23/2009)
2. Explaining the business value of iterative development
3. We have a huge budget and too much time to complete our business intelligence (BI) project

If you agree with the title of this post then you don’t need to read further, either your business intelligence (BI) project will be amongst the very few (less than 40%) that will not end in abandonment / failure or you do not have to worry that your project will drain your huge budget before slowly and gradually failing.

For everyone else, this article that presents the benefits of using an Agile approach to software development. This post is a follow up to my previous article called Nobody is interested in Agile where I wondered why so few articles explained the actual benefits (the WHY) behind Agile.

Why should organizations use an Agile approach for their Business (BI) Intelligence projects?

In clear business terms, using an Agile approach for the development of your BI solution will greatly increase your return on investment (ROI). Before reading further, you need to keep in mind that Agile will not address all of your software development issues but by empowering the people who are directly and indirectly part of your project team and by taking a pragmatic and realistic approach to the software development process, the implementation of Agile principles within your organization will address the most critical problems typically encountered by a software development project.

The following diagram summarizes the 7 key benefits of implementing an Agile approach for your business intelligence project. Each of the benefit is explained further below.

increased return on investment by using an agile approach

The intent of this post is not to describe the Agile principles nor to explain which practice should be applied in a specific context but to describe in clear business terms the benefits derived from these practices.

Increasing Productivity

The throughput of your development team will greatly increase once you implement agile methods. The associated benefits:

• Streamlined and improved face-to-face communications;
• Continuous performance improvement by retrospecting at the end of each short iteration;
• Progress is not measure by looking at compliance to a project plan but by evaluating by the quality of the output;
• Project management is shared by everyone on the team instead as the team self-manages and becomes fully accountable for the results and reporting mechanisms;
• Use of proven and innovative software engineering practices greatly improves quality;
• Increased ability to estimate efforts and time lines;
• Team productivity can be assessed and monitored over time which provides a predictable rhythm.

Meeting Time Commitments

Project time lines are typically very critical and as such implementing an Agile approach help meet time commitments.

• Reduce the overall scope into small manageable chunks and maintain focus on short term delivery instead of the final end date;
• Make the team’s progress visible to everyone with the use of defined reporting (burn down chart);
• Team has to demo working product at the end of each iteration;
• Estimating and scheduling is a collaborative process shared between the customer and the project team;
• Frequent delivery of software reduces the overhead of moving the application into production;
• End users are involved in defining the time lines instead of leaving the activity to the development team.

Increasing Quality

Quality is never negotiable so implementing Agile is beneficial to the project team.

• Team has to demo working product at the end of each iteration;
• Early detection and fixing of faulty components;
• Testing is not only done at the end of the project when the cost of fixing issues is much greater;
• Tests may be written before the development cycle begins and must be continuously be run throughout the duration of the project instead of waiting until the end.

Delivering on Requirements

Many projects are delivered without meeting the expectations of the customer. Using an Agile approach helps the team deliver on the requirements.

• Customer is part of the project team, defines the priorities and assesses the end results;
• Obtain continuous feedback throughout the development cycle and not only at the end;
• Team will deliver on the original requirements but if there are requirement changes, the team will provide opportunities for feedback and adapt to meet the changing requirements;
• Analysis and design is done at the beginning of each iteration and not only at the beginning of the project;
• Software should meet the current needs, not the needs that were defined months before.

Delivering Business Value

As opposed to emphasizing tools and processes, Agile focuses on delivering value to the organization.

• Teams work on value added tasks (as opposed to paper work and attending meetings);
• Teams delivers the highest value features first and avoid building components that will never be used by the customer;
• Focus on results (as opposed to process oriented or project plan driven);
• Time boxed to ensure compliance to defined time lines;
• Costs and benefits can be associated directly with the pieces of software being delivered, if the estimated cost for a component is greater than the anticipated value the component may not be developed;
• Reduces or eliminates non-critical activities;
• Raising issues and impediments early in the process reduces the cost of fixing them later on.

Improving Knowledge Sharing

As Agile relies on the concept of the team as opposed to the individual, knowledge sharing is a direct benefit.

• Prevents the fact that a single individual becomes the bottleneck for the project by emphasizing sharing of information and tasks amongst the team members.

Increasing Employee Satisfaction

Although not frequently mentioned, employee satisfaction is an important benefit of implementing Agile.

• Relationship between the project team and the customer are improved by developing trust and sharing knowledge throughout the duration of the project;
• Team self-organization and autonomy greatly improve team members’ morale;
• By inspecting and adapting their work processes, the team members grow their skills and motivation;
• Given the opportunity to play various roles on the development team, individuals increase their satisfaction by developing new skills.

As I mentioned in my previous post, there were very few articles that explained the benefits (the WHY) of implementing an Agile approach for software development project. Hopefully, this post will help you promote agility within your organization.

You might be interested in these related posts:

1. Improving ROI and Success Rate of your Business Intelligence Project
2. Explaining the business value of iterative development
3. Warning Business Users! People are Wasting your Budget

The un-impressive track record in the development of BI applications led us to conceive an improved approach to reduce the initial investment required to launch a BI project in order to deliver key performance indicators (KPIs) much more quickly and at lower cost.

The incremental development approach we came up with is aligned with the business priorities while allowing business users to further develop the knowledge of their business and better define their expectations.

As opposed to the traditional waterfall approach to building a BI application which is: plan the project, define the requirements, prepare the architecture, develop, test, move to production in a sequential manner, we propose to use an iterative approach during which we build layers of the entire data warehouse through iterations.

For example, instead of spending time in defining in a high level requirements, work with the end users to understand their business and the indicators they use to successfully manage it. Using an iterative approach, the objective of our process is to work backward starting with the required performance indicators back to the source systems.

An Iterative and Incremental Approach for BI Projects

As can be seen in the diagram, the intent is not to divide the project into phases where data is extracted from the source systems first, then move to a centralized repository and so forth. Instead of the traditional approach, we use a layer approach where each indicator is developed from source system to the presentation layer.

Although our approach requires a substantial paradigm shift for most people having experienced a traditional waterfall approach for BI
projects, the notions we propose are simple, logical and straightforward. This approach relies on AGILE principles and
more specifically on SCRUM practices that have been demonstrated to be successful in other types of initiatives. Our objective is to present them in a very concise manner in this post.

Our proposed approach to BI development can be summarized by the following principles:

• Segment the development of components into small iterations of 3 to 4 weeks and frequently deliver key performance indicators at the end of each iterations;
• Repeat the iterative process and use increments to add components to the BI architecture and infrastructure;
• Prioritize the development of the key performance indicators that will deliver the highest value to the organization first;
• Using a top-down approach going from aggregation to granularity, develop each KPI from end-to-end or from source systems to presentation layer;
• Avoid “analysis-paralysis” forces by using a “good enough” approach in documentation, architecture, modeling, project management
  and other overhead activities;
• Learn from each of the iteration and adapt the process appropriately to increase efficiency.

Our approach promotes a process that encourages frequent inspection and adaptation, a set of engineering best practices that allow for
rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals. With this
approach, the project delivers in small increments with minimal planning, rather than long-term planning.

You might be interested in these related posts:

1. Explaining the business value of iterative development
2. Explaining the business value of incremental development
3. Incremental Data Warehouse Development – The Only Way to Fly