Archive for January, 2009
My Rating: 6 / 10
A perfectly messy book!
A Perfect Mess: The Hidden Benefits of Disorder – How Crammed Closets, Cluttered Offices, and on-the-Fly Planning Make the World a Better Place is not intended to justify messiness, but the authors clearly demonstrate that over-organizing is an ineffective and costly endeavour that usually doesn’t bring as much benefits as is typically anticipated. In other words, in many instances the authors show that the cost of organizing is much higher than the resulting benefits. If you believe that being very organized is always best, you will be shocked when you read this book. Unfortunately if you are looking for reasons to be messy you may find that this book will support your condition.
Although this book was fun to read with plenty of anecdotes about the success of messy scenarios, it fails to provide valid comparisons for similar situations. For example, the book talks about situations where a positive outcome happen because of the surrounding mess but there is no replica of the same situation without the mess to determine if potentially a better outcome could have occurred.
Overall, the book was interesting with its many anecdotes and peculiar circumstances but the authors pushed it to such extend that the examples quickly became slightly exaggerated and inappropriate to demonstrate their point.
In my mind, the only conclusion to draw is to evaluate the need to organize and assess if the benefits call for the associated efforts. Use your judgement and determine how much organization is actually required and how much mess can be tolerated.
Sent: January-08-09 8:49 AM
[former colleague's name]
My Rating: 7 / 10
On the positive side, the book is a good discussion of how the principles of Lean Manufacturing apply to Software Development. The authors explain why the usual metaphor of software as manufacturing is not quite right and why the metaphor of Lean Manufacturing is something we can learn from and apply fairly easily to application development.
The authors demonstrate it has been a mistake to think of software development as being roughly analogous to manufacturing and that developing custom software is not like assembling cars. Along those lines software development is closer to product development and principles that work well in product design have a more straightforward application in software development.
The book presents 7 principles (Eliminate Waste, Amplify Learning, Decide as Late as Possible, Deliver as Fast as Possible, Empower the Team, Build Integrity In, See the Whole) and each chapter is devoted to one of the principles. I included a brief summary below.
On the flip side, this is not the first book you should read to learn about Agile software development as it assumes you clearly understand the underlying principles and the correlation to manufacturing can sometimes be difficult to relate to.
Example of elements that are considered waste:
- Anything that does not add value;
- Partially done work such as code that is not yet integrated;
- Extra processes such as unnecessary paperwork or documentation;
- Extra features that do not bring value;
- Task switching;
- Motion such as trying to find someone to get questions answered;
- Management related activities.
The authors demonstrate that design is a problem-solving process that requires discovering solutions through experimentation. A sequential development model does not usually provide for much feedback which is unfortunate because frequent feedback loops amplify learning.
Conventional wisdom in project management emphasizes compliance to cost estimate and defined schedule at the expense of achieving the overall business goals. Lean Software Development methods add much more control in between steps within the development process and increase feedback between activities as opposed to evaluating the output at the end of the process.
Decide as late as possible
Since people find it difficult to make irrevocable decisions when there is uncertainty present, a Lean Software Development approach recommends to decide on details as late as possible because doing otherwise would close opportunities that would be more suitable to the situation at hand. As such, an adaptive paradigm of delaying decision until uncertainty is reduced usually produces better results than a predictive approach.
Deliver as fast as possible
Deliver as fast as possible is a correlary to decide as late as possible since the faster you can deliver, the longer you can delay decisions. In addition, the authors suggest that the customers’ needs should pull the development efforts rather than have the schedule push the work. When put together, the whole system of software development short iterations is based on customers’ input at the beginning of each iteration.
With this approach, customers write-down description of the required features on index cards while developers estimate the efforts to deliver each feature. The developers then choose which cards they will develop.
Empower the team
The critical factor in motivation is by moving the decisions making power to the lowest possible level in an organization while developing the capacity of those people to make decisions wisely.
People suffer when they lack purpose. Intrinsic motivation comes from the work we do and for pride in workmanship and our ability to help customers. To help the team gain a sense of purpose, start with a clear and compelling purpose and be sure the purpose is achievable given the team access to customers then let the team make its own commitments. In this context, the management role is to run interference from skeptics away from the team.
Do not forget to deliver customer value while ensuring that systems components work together as a whole.
The proposed mechanisms documented in the book are:
- Running tests assume that the code is written;
- Don’t write more documentation, write more code;
- Don’t gather more requirements, show more progress by delivering components;
- Do work for an immediate real customer;
- Prototypes synchronize efforts towards a well understood short-term goal;
- Iteration results in a full working portion of the final product;
- Feature means to deliver meaningful business value;
- High priority features should be developed first to add value early;
- Development team only accept the amount of work for an iteration they can compete;
- Deliver on commitments;
- Teams need organizational support to achieve their objectives;
- Customers don’t know what they want at the beginning of the project;
- Deliver on top priorities to build credibility and trust;
- Develop the ability to respond to changing needs;
- Frequent builds to ensure components will integrate well and allow for testing.
In my opinion, the fundamental learning behind the book are:
- That it is critical to add value throughout the development process and not have to wait until the end of development;
- Empower your team members so they get the autonomy and authority to determine the right steps to deliver quality outputs;
- Ensure continuous learning in order to ensure innovation and improvement of the existing process.
As with other major changes, successfully implementing a lean software development approach requires change in the organizational habits and the underlying culture.
I received 7 magazine subscription offers in the last 2 days through the mail. Not 1 or 2, but 7. All of the magazines offer quality content and I could start receiving them within 6 to 8 weeks at a fraction of the cost (between 66% and 90% off the regular price). In addition, each of the magazine is offering me a gift: a pen with the magazine name on it, a laptop bag, a digital desktop clock, etc.
As I mentioned in an earlier post related to the same topic, I believe the golden era of printed magazines is behind us. With so much free content readily available on the Internet and with ever decreasing available time from reader, they need to rethink their business model.
As if I needed more reasons to throw these offers in my recycling bin, I realized how disconnected the people sending out these offers are. Would they be interested in receiving such bad gifts? What would they do with another laptop bag? Or with another cheap branded pen?
The only offer that made me think about subscribing was the one who would add hundreds of points to my frequent flyer account. It was the only offer that seemed actually directed toward me – the potential client. Had they offered a little more points, I would have fell for it even if I don’t need another magazine.
Bill Inmon made a statement with his post Incremental Data Warehouse Development – The Only Way to Fly.
My January 1st post called New Year’s Resolution was in line with his point that “end users cannot tell you what the actual system requirements are before the system is built. End users of analytical systems are constantly changing their minds. End users of analytical systems operate in a mode of discovery”.
The incremental and iterative approach allows end users to see what data they can get and how they can extract information, insight and knowledge out of the data to improve their decisions. After seeing and experimenting with their new BI tool, end users can then derive better requirements, hence the iterative approach.
By following an incremental and iterative approach “small part of the data warehouse is designed, programmed and populated. Then, another small part of the data warehouse is designed, programmed and populated, and so forth. A small, fast piece at a time is the way to build a data warehouse”. This process requires that the end users be an integral part of the development cycle which brings the benefits of ensuring the system does meet the end user’s expectations in addition to giving a sense of ownership to the end users as they were part of the development cycle.
Bill recommends the Data Warehouse Project Management book by Adelman and Moss which I haven’t read – yet. It’s now in my cart.
My Rating: 9 / 10
I read Agile Data Warehousing: Delivering World-Class Business Intelligence Systems Using Scrum and XP a few weeks ago and I thought the author did an amazing job at demonstrating how the Agile approach can be used for Data Warehousing projects. A such, I wanted to spend some time providing a more detailed summary of this book.
Agile Data Warehousing (ADW) is the application of two agile development approaches SCRUM and XP (extreme programming) – to the specific challenges of data warehousing and business intelligence.
The new approach is being proposed to address lamentable success rates of waterfall BI projects and as such ADW addresses the following problems:
- Waterfall software development historically yielded notoriously poor ROI;
- Customers are increasingly impatient to get BI benefits;
- BI cannot avoid tackling the unknown;
- Infrastructure problems cause problems late in the project;
- Business rules can also appear late in the plan due to changes in the environment;
- Project often take fatal risk in order to avoid a little rework;
- Requirements definition and review cycles consume excessive efforts;
- Documentation has a short shelf life;
- Standard project-management consumes large portions of the project budget;
- Complex methods and elaborate plan rarely perform well;
- BI need more time to acclimate the project;
- Customers need more time to learn about BI.
If good decisions allow a company to market its products or services more accurately, reduce its production costs or amplify its ability to fill orders, then a software engineering method leading to faster and better development of data warehouse offers much more than cost savings. The proposed approach is based on the following 4 Agile principles:
- Individuals and interactions over processes and tools;
- Working software over comprehensive documentation;
- Customer collaboration over contract negotiation;
- Responding to change over following a plan.
The principles behind the agile manifesto are known but they are repeated to ensure a common understanding.
- Our highest priority is satisfied customer through early and continuous delivery of valuable software.
- Welcome changing requirements even late in development agile processes are changed for the customers competitive advantage.
- Deliver working software frequently from a couple of weeks to a couple of months with a preference for the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals, defend the environment and supporting to entrust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors developers and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity is the art of maximizing the amount of work done is essential.
- The best architectures requirements and designs emerge from self organizing teams.
- At regular intervals the team reflects on how to become more effective than tunes and adjusts its behavior accordingly.
- Even the best approach needs to be cultivated and refined into a step-by-step method that teams can follow.
Naturally, achieving breakthrough performance in the BI development requires some significant changes:
- The physical environment developers work in.
- The role of the project manager.
- The way we do requirements.
- The way we do estimates.
- The way we do design.
- The way we code and tests.
- The way we document.
- The way we implement the system.
- The way we relate to other corporate departments such as information systems.
- The way we relate to other business stakeholders besides the project sponsor.
The five stages to migrate to an Agile approach are summarized below.
Stage 0: Co-located self organized team
- Co-locating the team members in a shared workspace surrounded by dry erase board, where they can work closely together, relying on light weight, verbal and diagrammatic communication for collaborative design and coding.
- Embedding the customer into the team as a product owner so that she can work closely with the developers to build the application the organization needs.
- Acting team members to organize their work in any way they prefer as long as they can transform a significant portion of the product owner’s request into potentially shippable code every 2 to 4 weeks.
- The team works in the minimalist environment for a few iterations, with the quality of his work review and approved by the product owner at the end of each iteration of development. The work product during this preliminary phase can be incomplete and lack polish.
- The customer does become fully involved and the team finds ways to meet pressing the deadline without wasting effort on that to be documentation.
Stage 1 – User epic decomposition and estimation skills
- This stage introduces the teams guiding product owner to several BI specific architectural notions so that the team begin acquiring a common language and so that many intertwined details of the full business intelligence implementation get addressed in an orderly fashion.
- The developers learn to estimate their work using the rim of the relative size of requirement. That technique which they will use to scope each development iteration.
Stage 2 – Release planning
- The team tests the full circle of the desired software release and the number of iteration it will take for it to deliver the remainder of the data warehouse project.
Stage 3 – Reference model and test-led development
Stage 4 – Pipeline deliveries squads
Stage 5 – Continuous integration testing
- By establishing an integration environment into which every module is integrated. As soon as a developer declared it done.
During the project, the team follow an iterative process to deliver incremental business value.
During the planning phase stakeholders represented in the team work through a list of business needs that might be addressed through new software development and agree upon which of them will be within the scope of the next release of the application.
Requirements are called users stories and together they make the release backlog.
The build phase of the release cycle involves many iteration or even shorter cycle the multi week development sprint in which the real coding work of the project will be done. The modules produced during each sprint are added to the deliverable of prior sprints until the team feels the entire collection represent an acceptable release which is then deployed.
The build phase of the release cycle consists of multiple development iteration called sprints. The sprints are likely structured time during which the team design and code the software modules needed.
The release cycle concludes with a review process where lessons learned can be identified and used to improve the effectiveness of the next turn of the cycle.
During this retrospective the team quantifies its velocity – the amount of work is successfully delivered within the time off of the sprint just completed. After a few sprints, the team velocity stabilizes into a predictable range allowing project folders to group the story in the release backlog using sprint-sized bracket and thereby forecasts how many time boxes it will take the delivered the entire release.
Long life to his new initiative.
If you will also be attending this summit and would like to share your thoughts on the topic of Agile Business Intelligence, drop me an email.