Monday, December 28, 2009

Agile Documentation

As a new theme to my blog, each week I will be pulling out one of the LEAN practice cards to talk about and discuss some of the concepts behind it as well as add in any colour commentary I may have based on my own individual project experiences.

The first card I thought would be interesting to look at is a Management Practice titled "Agile Documentation". Documentation is often overemphasized by some project teams or ignored by others depending on whether they subscribe to a waterfall or iterative based process. There is no clear cut answer on exactly "how much" documentation should be done as it is dependent on a variety of scaling factors.

Regardless of which development process your team may subscribe to, if we go back to the "why" behind documentation, I think applying Agile Documentation will help any team minimize documentation waste while also providing the benefits of documentation (yes, documentation is a good thing and even agile teams need to document as going agile is not an excuse to avoid documentation).

The LEAN Agile Documentation card states:



Document...
- When the business asks you to
- For clarity across teams working in different locations
- To establish appropriate business and technical context for the solution
- To define interfaces across systems
- To enable your next effort
- Informally
- To understand
- To communicate

Don't Document...
- To specify
- Because your process says so
- Unproven ideas (prototype it first)
- Without obtaining the structure from your intended audience
- Without considering maintenance and cost of ownership
- Implementation and design details that can be just as easily expressed in the solution itself

There are three main themes I want to pull out from this card that I think are worthwhile to discuss.

1) Document for an audience - Documentation requires time and effort and the business is paying for this just like they are paying for each feature built into the solution. Every project should contain estimates for time/effort (i.e. cost) towards documentation and this is something the business is paying for. If you look at many organizations and their development processes you will notice that there is often a large set of documentation that needs to be completed as part of the project. However, it's worth questioning does the business really need each one of these documents? Does the technology teams building and supporting the solution need these documents? If the team is documenting...

- Because your process says so
- Without obtaining the structure from your intended audience

Then it's likely the team is not documenting for the business. Generally the type of documentation I find invaluable to the business and the technology teams supporting the solution are operations and support manuals (e.g. run books), developer setup and build manuals, end user manuals and the delivery or release plan. This set may need to be expanded depending on the project.

2) Avoid documenting things that will change - Just like code and test cases, documentation is impacted by changes. To avoid paying the "rework" cost, hold off on documenting until a steady state is achieved (document at the last responsible moment) and stay away from implementation and detailed design details that will require frequent changes to the documentation.

3) Document to understand, communicate and establish contracts - It's important to recognize when the scaling factors impact your project and documentation is a great way to mitigate these risks and challenges. When your team is:

- Geographically distributed
- Large and difficult to manage
- Working in a complex business domain

Then documentation helps everyone understand the same language, communicate decisions and changes, and establishes contracts that helps each member understand how their work interacts with everyone else. The type of documentation I find myself often using to help the team navigate through these challenges are the high-level requirements (e.g. use case hierarchy) produced from the initial requirements envisioning sessions, ubiquitous language dictionary and high-level domain model, system context diagrams and bounded contexts, and system interfaces. An extremely effective practice that I often apply to documenting system interfaces or complex business domains is to use "executable requirements". Writing test cases as a form of documentation is a great way to precisely capture the details while also validating for correctness.

I have found these three themes combined with the bullet point checklists in the Agile Documentation practice card to be incredibly helpful in deciding what and when to document.

Wednesday, March 4, 2009

DSLs and the new Analyst tester

Domain specific languages (DSLs) are quickly gaining momentum and there is definite interest in how the industry can adopt DSLs in the requirements world. I think DSLs can act as a bridge to help cross the divide between an "analyst" and a "tester". While, I don't think the industry is ready to adopt DSL's in the large, there are several easy wins here that I think can be easily gained by the industry if we start small.

1) Get the analysts involved with delivery -it's time to accept that the old fashioned approach of making up requirements and throwing it over the wall has failed. Instead, embed the analysts with the developers and testers. Make them work with the team, jointly capturing requirements together and creating tests that validate that the system.

2) Executable Requirements - start writing automated acceptance tests. These acceptance tests are described one level up from "code" and can start looking like a natural language. Wikis like FitNesse (http://fitnesse.org/) are an excellent example

3) Behaviour Driven Development - the next step up from executable requirements is to embrace a DSL-like mindset to drive the design, development and testing. There are emerging frameworks that allow a tester or developer to write test cases in code that mirror the language of the business. See, http://behaviour-driven.org/

If we can get these three practices right, then I think we are half way up the hill towards understanding and using DSLs. Already there are many organizations and teams that are pushing the frontier with these practices. As these practices get adopted, the "analyst" will evolve into a capital A - "Analyst", lower case t - "tester" and the two roles will be merged into one.

Sunday, February 1, 2009

You know you are not doing Agile if

A common misconception I find many people are falling into when they are learning agile or using it for the first couple times is that they think Agile = Waterfall + Iterations. I'll call the second approach "Iterative Waterfall" for the purposes of this discussion.

While both models appear may similar (both are iterative and consist of requirements, design, coding ,and testing activities) there are significant differences that make them completely different animals. If we were to simply follow a waterfall based approach with small iterations then the flow of activities within an iteration would like look this:


As everyone is probably familiar with, this is the mini-waterfall (waterfalls within an iteration). The developers and customers follow a sequential set of activites as prescribed by the waterfall model. The flow of work is predictive and follows a set path. While this model is iterative it is not agile. Why? The reason is because an agile model will support continuous learning and refinement not only between iterations but also within iterations. It does this by intertwining the requirements, designing, coding, and testing together into an unpredictable flow of work.



The developers and customers jump back and forth from one activity to another in an unpredictable manner. There is no set path. Instead, as they start implementing a user story or use case, they may discover new insights that require them to revisit and redefine their initial set of requirements which require them to redesign the solution, create or rewrite tests and code. Iterative learning and change is enabled within each iteration. This allows the developers and customers to quickly and easily react to change and stay agile.

So, if you are doing agile right now ask yourself which model your using and if you are not doing the second model then I strongly urge you to try it out. You will find that the agile practices fit better and allows your team to fluidly move from one activity to another. Remember, you are not doing agile if your team isn't continuously learning and able to react to the new knowledge.

Friday, June 13, 2008

A small agile success story

I recently wrapped up a small development project where our client engaged us to design and build a customized claims administration application.

It was a high priority tactical solution driven by legal and accounting requirements that kick-started a claims process that was rolling out in three weeks. The custom claims administration application needed to be delivered within this three week timeframe to administer the process. It was high risk since the application needed to be accurate, stable and responsive since claim administers interacted with it in real-time while conversing with claimants.

In order to deliver against this aggressive timeline we needed the following:

- lightweight process that could respond to rapid changes (requirements definition was in-scope)
- a process that was easy to track progress against the goal
- a process that enabled frequent and reliable estimation for planning (we needed to know right away if we were falling behind)
- a process that would allow us to communicate to management and stakeholders on our progress and get their buy-in when we push back on features to control scope (it's amazing sometimes how non-technical resources underestimate effort, and how difficult it is to communicate realistic effort)

We rolled out a flyweight agile process driven by user stories and 1 week iterations. This would allow us to:

- capture requirements in terms of user goals and value (user stories are great for this, and easy for non-technical resources to understand)
- prioritize stories with management and stakeholders
- estimate each user story (we used ideal days)
- create a release plan and iteration plan based on user stories (functionality and value instead of tasks)

Our first goal was to prioritize the absolutely must have user stories which would make up the first release. This release consisted of three rapid iterations targeted to launch on the three week deadline. Lower priority user stories would be rolled out in a subsequent release. Our release and estimation plan was updated weekly which allowed us to easily track and communicate progress. We also tracked our velocity and noticed after two iterations we needed another resource to meet our plan. It also allowed us to re-prioritize weekly and we evolved our release plan to break out user stories into two staggered release.

User stories were great since they were low ceremony, light on documentation and less susceptible to change since details are captured at the last moment via conversations. It really enabled agility since we were not afraid to throw away or change requirements since we placed so little investment into them (high value for low cost).

As a result we delivered on time and the project was a success.

Overall, I think an agile based approach enabled our success. Rapid feedback, frequent communication, and lightweight artifacts allowed us to respond to change and adapt to succeed.