Gartner studied the market and attempt to predict trends in their latest report: Predicts 2010: Agile and Cloud Impact Application Development Directions.

As organizations seek to improve productivity and reduce application operating and maintenance costs, we will continue to see an evolution of software development tools, platforms and practices. To take advantage of this, organizations must shift structures and practices while embracing new technologies — a challenging proposition.

Gartner’s analysts (Thomas Murphy and David Norton) predict that by 2012 “agile development methods will be utilized in 80% of all software development projects”. The authors explain that although Scrum will continue gaining in popularity over the coming years, organizations will not be successful in their transition unless they move toward a team-focused culture. As was mentioned in their previous report, very few organizations use a pure-Scrum approach and most rely on an hybrid approach (waterfall and Agile).

The report highlights that organizations struggle to implement true collaboration in the context of globally distributed teams. A situation that has amplified in recent years with outsourcing and off-shoring of software development projects.

In the other hand, the report confirms that teams who have successfully moved to Agile do see productivity improvements especially in “the flexibility of the development team to respond to shifting requirements”. This is especially true for web-based application developments where rapid responses to a changing environment is critical.

The authors point out that organizations need to properly invest in such a transition in order to achieve success.

Organizations that do not make use of key agile practices and do not invest in training and supportive tools’ infrastructure will find that a shift to pseudoagile, while potentially delivering a short-term productivity bump, will end in long-term declines in quality and productivity (…) the promise of four times the improvements in overall productivity has been and will be achieved by select organizations.

Gartner’s report highlights that “development organizations have been making a shift toward agile methods, but this is still slow to move beyond development, and often is a mixture of waterfall practices utilizing an agile or iterative project cycle”. The authors also recommend to “look for opportunities to utilize agile development practices, but recognize that it requires changes and commitment on the part of business and IT”.

Gartner concludes with a few recommendations to help organizations maximize their return from an Agile transition.

• Recognize the cultural changes that are at the heart of agile.
• Don’t allow agile excitement to drive cowboy-coding practices.
• Agile requires discipline.
• Recognize that scrum is only a partial solution, and focus on a collection of practices.
• Find tools that enable collaboration and help automate repeatable, consistent practices.

Related documents:

• Don’t Let Short-Term Agile Create Long-Term Pain
• The Current State of Agile Method Adoption

As a follow up to my previous post, this second post in a series of 4 short articles written in collaboration with my colleagues Stéphane Lécuyer, Jean-René Rousseau, Sylvie Trudel, Joël Grenon, and Eric Laramée, addresses the impact an Agile transition typically has on some of the traditional software development roles: the project manager, the architect, the business analyst, and the QA specialist.

Based on our experience, here are possible strategies to properly map the traditional roles to the three roles defined by Scrum.

The Project Manager

Traditionally, the project manager is responsible for determining who, what, and when activities need to be performed and then to ensure the team complies with the plan that was prepared to meet the budget, time and scope constraints.

With the traditional approach, project management is based on compliance with the plan while Agile and Scrum propose a different approach where maximizing the business value is the main vector of project management. Under this new approach, the product manager needs to collaborate with the team members and delegate to them some of his traditional responsibilities since they will determine who does what, and when within the constraints of the project.

In this context, the role of the Scrum Master is to enforce the process and seeks to build an efficient self-organized team. To the question “do we still need a project manager in Agile?”, experience shows us that in most organizations, the answer is yes.

The need for accountability, regulatory compliance and alignment with the framework and IT governance are not covered by the role of the Scrum Master and as such remain the responsibility of the project manager.

However, the project manager needs to adapt its management style and use leadership rather than authority with the team to get things done. In the context of a multi-team organizational structure, the presence of a project manager is also valuable, where he is coordinating the teams and the synchrony between them and between entities external to the project teams.

From our experience, some project managers are more willing to become product owners while others will feel challenged by the role of Scrum Master. In the end, it will be the responsibility of the organization to determine how to redefine the roles and their associated responsibilities.

The Architect

Similar to the project manager, the architect is known to play a different role post-transition compared to that required in traditional development teams. He must act as a consultant to the teams and provide them with the necessary support instead of dictating the rules and guidelines to be followed. The architect should also be familiar with the concepts of emerging architecture, where just enough architecture is planned to allow the team to innovate and find the optimal solutions.

The architect then acts as a catalyst for sharing best practices within the organization. Here is a list of practices summarizing the changes of behavior for the architect:

• Is an active member of the development teas, helping to build the right software and acting as consultant;
• Does not attempt to predict the future, he provides a coherent vision but knows that tomorrow’s problems will be easier to solve tomorrow;
• Is changing its architecture in an incremental way, leaving room for emergence;
• Does not seek to document everything to perfection, he focuses on a few relevant diagrams and documents the best practices based on his audience;
• Seeks to validate its concepts through concrete experiences.

Once again, although the role of the architect does change after an agile transition, it remains an important role to be filled.

The Business Analyst

The business analyst is another role that seems neglected by Scrum. To ensure close collaboration between the team and the Product Owner, Scrum ensures that the necessary elements are effectively communicated directly to the team without a formal and complex documentation. However, to ensure continuity of information, we know that functional documentation that is adequate and representative of the software to be developed is essential.

The business analyst becomes a valuable contributor to the Product Owner. The responsibilities of the business analyst are as follows:

• Supports the Product Owner in gathering and writing the required stories;
• Does just enough analysis for the functionality to be carried out during the next iteration;
• Prepares and updates documentation used at the end of each iteration;
• In collaboration with the QA Specialist, helps determine the quality assurance strategy.

In a multi-team context, the business analyst may be called upon to play the role of Product Owner. He then becomes responsible for core components of the product within the various sub-teams.

The Quality Assurance Specialist

Quality is a fundamental concern in Agile project management and each iteration should produce an increment of quality software. To do this, we recommend incorporating a quality assurance specialist within the Scrum teams, and right from the start of the project. A QA specialist assigned to a Scrum team has the following responsibilities:

• Participates in planning sessions to raise issues relating to quality;
• Helps clarify the definition of “Done”‘;
• Prepares plans for acceptance testing;
• Validates the increments of product delivered.

Other Roles

As will be presented next week in “Part 3: Impact on the functional and people managers”, managers also get impacted by an Agile transition.

If you have been reading about Agile for a while and are interested in a transition or if you have already initiated a transformation, you have previously heard all the benefits that Agile can bring to your organization but …

Are you aware of the impacts such a transition will have on your organization? On your team? And on yourself? Would you know how to deal with these impacts?

If you believe that implementing Agile within a company simply means reducing documentation, standing up during daily meetings, using whiteboards and post-it notes, and getting rid of the project manager, you will certainly be shocked to see how profound the changes can be.

In a series of 4 short articles written in collaboration with my colleagues Stéphane Lécuyer, Jean-René Rousseau, Sylvie Trudel, Joël Grenon, and Eric Laramée, we aim to highlight some of the most common (and rarely described) impacts an Agile transition can have on an organization. The articles will be published weekly and will cover the following 4 impacts.

• Part 1: Impact on the organization
• Part 2: Impact on some of the traditional roles
• Part 3: Impact on the functional and people managers
• Part 4: Why a coach is useful

Adopting Agile practices is not a trivial change; it requires support and time to become effective. The use of external coaches, training materials, and internal support groups can greatly increase the speed and success of adoption. - Forrester Reports “Agile Development: Mainstream Adoption Has Changed Agility”.

Many organizations rely on external consultants to help them successfully transition to Agile. Others initiate a small transition after having researched the best practices. Having gained experience from the implementation of Agile within organizations over the last 8 years, we can attest that the impacts related to the establishment of an Agile development approach affect many areas in the organization. Through our experience, we have prepared a high level description of potential impacts you may want to anticipate before getting deep into your transition.

Next week’s post will address more specifically to impact on the role of the project manager, the architect, the business analyst, and the QA analyst.

As part of a market research for one of our customer, I came across this report published by Gartner in December 2008.

As the pace of agile adoption increases, development organizations must understand the different levels of agile maturity. CIOs and product and development managers need to assess where they fit on the maturity scale, and which level offers the biggest return in their organizations.

The report presents the 6 levels (from 0 to 5) of Gartner’s Agile Maturity Model and corrects a few myths.

Agile adoption and penetration rates are being overestimated. Although the number of companies that are adopting agile practices is, indeed, reasonably high, most organizations use agile in a very small percentage of their overall work.

An agile maturity framework is necessary to help make the case for adoption, process improvement and benchmarking.

Current adoption rates for agile and iterative methods are between 15% and 25%, when taking into account penetration and maturity, with waterfall still the dominant approach. The pace of agile adoption is increasing.

The report concludes that :

As part of an agile readiness assessment, IT development organizations should access their current agile practice maturity at technical, project management and organizational levels. Practices should be assessed for, among other things, their effectiveness and adoption levels in the organization. Adoption should follow initial pilots, and should normally be Level 2, with the aim of developing a consistent set of agile practices at Level 3.

RECOMMENDED READING

• Maturity Model Overview: Application Organizations

A few days ago, we invited representatives from a potential customer over to visit our office. They are seriously considering a transition to Agile but some of the managers had questions with regards to what an Agile work space could look like. The potential customer is a large insurance company and like most insurance companies, people working there are used to a traditional (very traditional) work space. We could see they had some reservations about the open-concept before coming for a visit.

Their visit lead me to wonder what other Agile work spaces could / should look like, so I came up with the idea of launching a friendly contest…

The “Best Agile Work Space” Contest

I invite you to email me a picture of your Agile work space (martin [at] analytical-mind.com). In the spirit of sharing best practices and getting ideas from each other, I will post your pictures and your company’s name for people to get inspired. You can also share with everyone what makes your work place the Best Agile Work Space. We’ll even ask people to vote!

Let the contest begin to determine the “Best Agile Work Space“. Tell your friends to email their pictures.

Examples of other Agile Work Spaces found on the web

Windows are often a scarce commodity and are doled out to an organization’s favored employees. One of the nice things about an open workspace is that windows are shared. Even if the view is only of our parking lot and can only be seen across three messy desks, at least I can see the window and some natural light - The Ideal Agile Workspace | Mike Cohn’s Blog – Succeeding With Agile®.

Our New Agile Workspace - Our New Agile Workspace on Flickr – Photo Sharing!.

I started to respond in his comments and then remembered that it would be better to capture our workspace on video to share with others.  I am hoping other agile shops will do the same.  We are always eager to see how others are doing things so we can continue to improve - Attempting to Achieve the Ideal Agile Workspace | Derek Neighbors.

Ward Cunningham among others was a big influence early on in making it happen.  The patterns & practices team workspace is optimized for agile development practices.  The workspace features writeable walls, configurable workspace, speaker phones, projectors, focus rooms, and a customer room - Shaping Software » Blog Archive » Microsoft patterns & practices Agile Workspace Tour.

Agile consultant

You can find more information on our web site.

“Pyxis helps software development companies to become places where results, quality of life, and fun coexist sustainably.”

Being first and foremost an example of what it proposes to its clients, Pyxis is seeking Agile consultants wishing to evolve in a unique organizational structure relying on autonomy, initiative, and collaboration.

The Agile consultant will carry out projects at the clients’ site. He will act as a coach or ScrumMaster in organizations wishing to adopt or improve Agile development approaches.

At Pyxis, consultants are software development buffs. They are pragmatic and they like to work with people from various disciplines.

Main tasks and responsibilities

• Carry out Agile coaching or transition projects at the clients’ site
• Help organizations with the implementation of Agile development approaches
• Act as a ScrumMaster within development teams
• Coach ScrumMasters to help them familiarize with their role
• Participate in projects at Ensemble, our development center
• Remain at the leading-edge of Agile development practices and contribute to their spreading

Requirements

• Diploma in computer engineering, computer science, or business computing, or equivalent experience
• Over 7 years of experience in IT, including 2 in Agile software development
• Broad experience in traditional or Agile project management
• Mastering software development processes
• Knowledge of main development methods
• Sound knowledge of Agile values, principles, and practices such as Scrum and XP

Assets

• Knowledge of software engineering practices such as TDD and DDD
• ScrumMaster certification from the Scrum Alliance

Soft skills

• Dynamism
• Autonomy and versatility
• Team spirit
• Facilitator’s skills
• Diplomacy and ability to resolve conflicts
• Clear understanding of a company’s business needs, business processes, and organizational dynamics

Remuneration

Remuneration based on the value you bring to Pyxis.

In their recently released study “Agile Development: Mainstream Adoption Has Changed Agility“, Forrester reports that “35% of respondents stated that Agile most closely reflects their development process”. The report is based on Forrester’s/Dr. Dobbs Global Developer Technographics Survey, Q3, 2009, which surveyed 1298 application development professionals.

In the past few years, Agile processes have not only gained increasing adoption levels; they have also rapidly joined the mainstream of development approaches. And while more organizations are adapting to Agile conventions, Agile is also adapting to the workplace. Perhaps the clearest sign of the mainstreaming of Agile is the abandonment of orthodoxy: Teams are puzzling out the mix of methodologies and combining them to fit within their organizational realities, blending Agile and non-Agile techniques and practices to create a hybrid methodology that fits larger organizations. Other changes, such as new team dynamics and the redefinition of roles such as the business analyst, show the genuine force behind Agile adoption. It’s time for software development professionals to stop sitting on the fence where Agile is concerned. According to those who have successfully adopted Agile, the benefits are well worth the effort, and with the recent dramatic increase in Agile adoption, the probability of working in or with an Agile team has increased for everyone.

The report looks at Agile Adoption from the following perspectives:

• Agile Adoption Goes Mainstream
• Teams Are Changing To Support Agility
• What it means
• Agile Becomes Mainstream — But Not Without Some Changes

And then offers a recommendation to get the most out of the transition: “App Dev Professionals Should Blend Agile Methods To Meet Their Own Needs“.

Agile methods encourage more-collaborative development than do traditional approaches, and many developers who have shied away from formal development methods in the past — believing them to be the province of “management” — have embraced Agile as a “formal” development process.

The report shows that of the various methodology, Scrum leads as the most adopted Agile methodology with 11% of the organizations reported using this approchach.  Other Agile approaches used are: Agile Modeling (6%), Feature-driven development – FDD (3.8%), Test-driven development – TDD (3.4%), eXtreme Programming – XP (2.9%), Lean development (2.1%), Microsoft Solutions Framework – MSF for Agile (1.8%), Agile Data Method (1.6%), Adaptive Software Development – ASD (1.3%), Six Sigma (0.9%), Crystal (0.3%), Behavior-driven development – BDD (0.2%) and Dynamic Systems Development Method – DSDM (0.2%).

When it comes to selecting an Agile methodology, Scrum is the overwhelming favorite – claiming it is simple, practical, and popular.

The report shows that when organizations select a development approach, they do so in the context of their organizational priorities and characteristics. Each organization carefully selects the approach that will best address their weaknesses rather than implementing Agile methodology for its own sake. In addition, participants considered the engineering processes to be critical to the success of the transition.

Teams do not usually implement all of these techniques simultaneously; even in the most mature Agile adoptions, teams pick the techniques that work best for them (…) This variation in the adoption of Agile components indicates that teams are more concerned with making sure they are working well together and producing high-quality software than with changing their software engineering process.

Perhaps the most important aspect of Agile’s entrance into the mainstream is the way that teams pragmatically mix methodologies. Instead of sticking strictly to a particular Agile orthodoxy, teams cherry-pick Agile methods, often including non-Agile techniques in the mix as well

Overall, the report clearly shows that organizations who have transitioned to an Agile approach use a pragmatic strategy and adapt the methodology that are best suited for their environment and context. While only 27% stick to a particular methodology, 63% mix different methodologies or mix Agile with non-Agile methodologies.

In conclusion

Agile adoption is a reality. Organizations across all industries are increasingly adopting Agile principles, and software engineers and other project team members are picking up Agile techniques. While historically, management has owned “process,” the adoption of Agile methods has pushed ownership into the hands of team members — many of whom have traditionally been skeptical of process and methodology. Broad Agile adoption requires careful consideration; a strong Agile adoption strategy should include:

• A support plan. Adopting Agile practices is not a trivial change; it requires support and time to become effective. The use of external coaches, training materials, and internal support groups can greatly increase the speed and success of adoption.
• Flexible adoption models. One size does not fit all.
• A focus on team empowerment. It sounds really easy, but this is about more than just saying that teams are empowered.
• A tool strategy. A single team in one location working alongside a customer may be able to work without any electronic tools, but as organizations scale and teams become more distributed and part of much larger releases, Agile methods benefit greatly from tools.

Related Research Documents

• “From Agile Development To Agile Engagement” - May 6, 2009
• “Ensure Success For Agile Using Four Simple Steps” - March 24, 2009
• “Make Agile Lean To Boost Business Impact” - December 17, 2008

Like most modern Homo sapiens, when you hear Agile Leadership or Agile Management, you think of:

• [if you are outside the business world] A business-person who can use a combination of balance, coordination, speed, reflexes, strength, endurance and stamina to achieve his objectives;
• [if you are inside the business world but outside the information technology field] A person who has the capability of rapidly and cost efficiently adapting to changes in an attempt to deliver on his objectives;
• [if you are inside the information technology field] A person who manages a software development team who uses methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams with the objective of delivering value.

I apologize if you are outside the business world because this is not the perspective I wish to cover. For people in the other two categories, you are partially right.

I attempted to define Agile management and see how I could apply Agile principles to management a while ago and since then, I have been able to piece the puzzle together. Agile Leadership requires less technical knowledge than its cousin but it heavily relies on the same principles.

A high level view of the model

If you have been reading my blog for a while (thank you!) and even if you haven’t, you will realize that I have been covering various parts of this model already:

People: The people dimension covers all aspects of competencies, motivation, culture, collaboration and communications that enable the organization to achieve its business objectives. While every effort is directly or indirectly related to people, this perspective focuses primarily on the ability of individuals to contribute to the achievement of objectives.
[related tags: 360-degree feedback, coaching, collaboration, community, decision making, feedback, leadership, management, organizational structure, people management,servant leadership]

Processes: The process dimension aims to define the working methods and approaches to be followed in carrying out tasks in line with the overall objective of delivering business value.
[related tags: agile, agile management, scrum]

Tools: The technology dimension covers the various tools and technologies that support the organization in achieving its business objectives.
[related tags: none, I haven't covered this dimension]

Value: The value dimension covers the business capacity to effectively deliver value within the appropriate time. The delivery of value is the fundamental purpose of the organization.
[related tags: ROI]

As you can see, I have mostly covered the People dimension of the model while I have purposely left the Tools section un-covered. The reason for this is that there are already thousands of web sites on the topic of Agile and technology.

In an upcoming series of blog posts, I will present a more detailed perspective of what Agile Leadership truly means based on our experience. Stay tuned…

On the importance of creating the right organizational culture (Thanks to Andrew)

By the time we got to 100 people, even though we hired people with the right skill sets and experiences, I just dreaded getting out of bed in the morning and was hitting that snooze button over and over again Corner Office – Tony Hsieh of Zappos – Celebrate Individuality – Question – NYTimes.com.

On why an Agile approach is better suited to deliver value (Thanks to Alfonso)

Most organizations that depend on software are struggling to transform their lifecycle model from a “development” focus to a “delivery” focus. This subtle distinction in wording represents a dramatic change in the principles that are driving the management philosophy and the governance models – Improving Software Economics

On the meaning of Agile transformation for managers

What many people mistakenly do is equate agile project management with doing more work, with less documentation and fewer people. Although the premise is to get more done in a more favorable way, I have never met a team that could successfully implement agile principles without having to slow down first - VersionOne – Agile Adoption For Managers.

On the fact that the true value of an organization is not mapped via its organizational chart

But it’s not the fact that you have many more boxes and lines that I’m most envious of.  It’s your “white space” I want – Oh, Yeah? Well, My Org Chart is Bigger and More Beautiful Than Yours!

On the need to manage self-organized teams when required

The interesting thing is, the further we go into agile management territory the less typical the managerial job we expect. Teams are self-organizing and cross-functional, and sometimes we think a manager should just get out of the way. By the way, surprisingly often this is exactly the best choice. But whenever one of the asshole-moments is needed, it is time to show up and do what has to be done. Otherwise the atmosphere starts rotting as people wait for someone who will fix things. Someone who will do something about this guy adding a new technology every time he reads some nice article. Someone who will deal with that lass taking a few days off because she doesn’t really care about the project being late and the team working their butts off to get back on the right track. That’s always a job for a manager, and a harsh one, no matter how self-organized the team is - Good Managers Sometimes Have to Play Assholes – NOOP.NL.

If you are using Scrum and Agile within your organization, you already know about the daily stand-up meeting and the value its brings to the team. Many organizations who have not fully adopted Scrum still find the stand-up meeting to be extremely useful when done properly – but this is not the objective of this blog post.

We have just released a really neat ipod touch app – the _agilely Timer. No, this is not a shameless plug but a way to help people in need. As part of the Agile Tour, Pyxis has released a timer application that allows you to efficiently facilitate daily stand ups, roundtable discussions and manage timeboxes. For only $1.99, this is a great way to help FIAN since all revenues will be donated to this organization that “fights hunger with human rights”.

Go ahead, get this neat app and stand-up for something today.

Buy it now - only $1.99

_agilely Timer to support FIAN.org

For only $1.99, help fight hunger

ipod touch and iphone timer application

Want to know more, you may be interested in this blog post in French or the English version translated by Google.