Ch7 21 bi

Introduction

Agile methodologies<ref>[1]</ref> are software development techniques based on software and management principles that have been distilled over the past few decades. The primary differentiating factor between Agile methodology and traditional software development methods is the fluidity that the Agile methodologies inject in all stages of software development.

Agile software development is an umbrella term that encompasses various development techniques and philosophies, however, there are certain core characteristics that all of these share. The key features of Agile development are that it is based on iterative and incremental development encouraging an evolution of the requirements and solutions. Agile software development teams also tend to be more fluid, self-organizing and are often cross-functional. Agile software development at the core of it, strives to break the rigid boundaries that traditional software development techniques introduce.

History and the Agile manifesto

Historically, software development has followed the waterfall model<ref>http://en.wikipedia.org/wiki/Waterfall_model</ref> with clear delineation of the various phases such as gathering requirements, design, implementation, verification and maintenance, with very little cross communication between these phases. This rigidity built into this model doesn't always reflect the real world requirements of software development, for e.g., the requirements could change after the design has been completed and a failure to accomodate the new set of requirements is often considered a failure of the project. This has resulted in a significant percentage of software development projects being delayed and in some cases being completely stopped.

As a reaction to the failure of the traditional software development models, quite a few new techniques started to develop in the late 1990's. They all emphasized close collaboration between the programmer team and business experts and a built in ability to be able to deal with and manage change. The core ideas behind these loosely defined methods, referred to as light-weight software development or Agile methods, were finally crystallized into a set of principles, i.e., the Agile manifesto.

The Agile Manifesto<ref>[2]</ref> reads, in its entirety, as follows:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Characteristics and Principles

While there are many specific agile development methods a common thread runs through all of them. The primary focus is on process adaptability throughout the software development life-cycle. This is achieved by constraining the development to short, time-contained increments. During each of these incremental development periods, there is a focus on a teamwork and collaboration with routine and formal daily face-to-face communication among team members.

Another key aspect of any agile methodology is the team composition, which is often cross-functional and self-organizing with traditional corporate hierarchy not allowed to get in the way of team formation. A cross-functional team is more reflective of the real-world in that all the stake holders in the project are allowed to influence the development process in a continuous manner throughout the life cycle of the project. This allows for adjustment in the goals of the project on a more `real-time' basis. Most agile teams include atleast one customer representative

Agile development emphasizes working software as the primary measure of progress. This, combined with the preference for face-to-face communication, produces less written documentation than other methods.

Some of the key guiding principles of Agile development are:

1. Active involvement of the end-user : This results in the requirements being clearly communicated and understood (at a high level) at the outset. Requirements are prioritized appropriately based on the needs of the user and market and can be clarified on a daily basis with the entire project team, rather than resorting to lengthy documents that aren’t read or are misunderstood. Emerging requirements can be factored into the development schedule as appropriate with the impact and trade-off decisions clearly understood. As the product goes through increments, it is easy to track if it continues to meet the user requirements and timely decisions can be made, about features, priorities, issues, and when the product is ready.

2. Requirements evolve but the timescale is fixed : Agile methodologies embrace change as part of the development process, however it is also key to make sure that as the requirements morph the timelines don't slip. So to include a new requirement, or to change a requirement, the user or product owner must remove a comparable amount of work from the project in order to accommodate the change.

3. A collaborative & cooperative approach between all stakeholders is essential : Agile development relies on close cooperation and collaboration between all team members and stakeholders. Agile development principles include keeping requirements and documentation lightweight, and acknowledging that change is a normal and acceptable reality in software development. This makes close collaboration particularly important to clarify requirements just-in-time and to keep all team members on the same page throughout the development.

4. Capture requirements at a high level; lightweight & visual : Agile requirements are ideally visual and should be barely sufficient, i.e. the absolute minimum required to enable development and testing to proceed with reasonable efficiency. The rationale for this is to minimise the time spent on anything that doesn’t actually form part of the end product.

5. Develop small, incremental releases and iterate : This reduces risk since we one has a clear visibility of what’s completed to date throughout a project. It also allows one to deliver value sooner, being able to release the product whenever it’s deemed good enough, rather than having to wait for all intended features to be ready. Developing iteratively allows more flexibility since we can choose to change direction or adapt the next iterations based on actually seeing and using the software.

6. Complete each feature before moving on to the next : Features developed within an iteration, should be 100% complete by the end of the iteration. In agile development, make sure that each feature is fully developed, tested, styled, and accepted by the product owner before counting it as finished.

7. Testing is integrated throughout the project lifecycle : Agile development does not have a separate test phase as such. Developers are much more heavily engaged in testing, writing automated repeatable unit tests to validate their code. Apart from being geared towards better quality software, this is also important to support the principle of small, iterative, incremental releases

There are quite a few agile software development methods :

• Agile Modeling<ref>http://en.wikipedia.org/wiki/Agile_Modeling</ref>
• Scrum <ref>http://en.wikipedia.org/wiki/Scrum_(development)</ref>
• Agile Unified Process <ref>http://en.wikipedia.org/wiki/Agile_Unified_Process</ref>
• Dynamic Systems Development Method <ref>http://en.wikipedia.org/wiki/Dynamic_Systems_Development_Method</ref>
• Essential Unified Process <ref>http://en.wikipedia.org/wiki/Essential_Unified_Process</ref>
• Extreme Programming <ref>http://en.wikipedia.org/wiki/Extreme_Programming</ref>
• Feature Driven Development <ref>http://en.wikipedia.org/wiki/Feature_Driven_Development</ref>

Some of these such as DSDM are stand alone, i.e., they encapsulate all the agile principles while others such as extreme programming and Scrum focus on parts of the Agile process and needed to be complemented with other methodologies to be complete. We will focus on a couple of these methodologies and look at some of the available Agile tools.

Scrum

Scrum is a framework for developing and sustaining complex products. Scrum is a process framework that has been used to manage complex product development since the early 1990s. Scrum is not a process or a technique for building products; rather, it is a framework within which you can employ various processes and techniques.

Three pillars uphold every implementation of Scrum: transparency, inspection, and adaptation.

• Transparency : Significant aspects of the process must be visible to those responsible for the outcome. Transparency requires those aspects be defined by a common standard so observers share a

common understanding of what is being seen.

• Inspection : Scrum users must frequently inspect Scrum artifacts and progress toward a goal to detect undesirable variances. Their inspection should not be so frequent that inspection gets in the way

of the work. Inspections are most beneficial when diligently performed by skilled inspectors at the point of work.

• Adaptation : If an inspector determines that one or more aspects of a process deviate outside acceptable limits, and that the resulting product will be unacceptable, the process or the material being

processed must be adjusted. An adjustment must be made as soon as possible to minimize further deviation.

Scrum consists of Scrum Teams and their associated roles, events, artifacts, and rules. Each component within the framework serves a specific purpose and is essential to Scrum’s success and usage.

The Scrum Team

The Scrum Team consists of a Product Owner, the Development Team, and a Scrum Master. Scrum Teams are self-organizing and cross-functional. Self-organizing teams choose how best to accomplish their work, rather than being directed by others outside the team. Cross-functional teams have all competencies needed to accomplish the work without depending on others not part of the team. The team model in Scrum is designed to optimize flexibility, creativity, and productivity.

Scrum Teams deliver products iteratively and incrementally, maximizing opportunities for feedback. Incremental deliveries of “Done” product ensure a potentially useful version of working product is always available.

The Product Owner

The Product Owner is responsible for maximizing the value of the product and the work of the Development Team. How this is done may vary widely across organizations, Scrum Teams, and individuals. The Product Owner is the sole person/group responsible for managing the Product Backlog. Product Backlog management includes:

• Clearly expressing Product Backlog items
• Ordering the items in the Product Backlog to best achieve goals and missions
• Ensuring the value of the work the Development Team performs
• Ensuring that the Product Backlog is visible, transparent, and clear to all, and shows what the Scrum Team will work on next
• Ensuring the Development Team understands items in the Product Backlog to the level needed.

The Product Owner may do the above work, or have the Development Team do it. However, the Product Owner remains accountable. The Product Owner’s decisions are visible in the content and ordering of the Product Backlog. No one is allowed to tell the Development Team to work from a different set of requirements, and the Development Team isn’t allowed to act on what anyone else says

The Development Team

The Development Team consists of professionals who do the work of delivering a potentially releasable Increment of “Done” product at the end of each Sprint. Only members of the Development Team create the Increment. Development Teams are structured and empowered by the organization to organize and manage their own work. The resulting synergy optimizes the Development Team’s overall efficiency and effectiveness. Development Teams have the following characteristics:

• They are self-organizing. No one including the Scrum Master, tells the Development Team how to turn Product Backlog into Increments of potentially releasable functionality;
• Development Teams are cross-functional, with all of the skills as a team necessary to create a product Increment;
• Scrum recognizes no titles for Development Team members other than Developer, regardless of the work being performed by the person; there are no exceptions to this rule;
• Individual Development Team members may have specialized skills and areas of focus, but accountability belongs to the Development Team as a whole; and,
• Development Teams do not contain sub-teams dedicated to particular domains like testing or business analysis

Scrum prescribes four formal opportunities for inspection and adaptation:

• Sprint Planning Meeting
• Daily Scrum
• Sprint Review
• Sprint Retrospective

The Scrum Master

The Scrum Master is responsible for ensuring Scrum is understood and enacted. Scrum Masters do this by ensuring that the Scrum Team adheres to Scrum theory, practices, and rules. The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which aren’t. The Scrum Master helps everyone change these interactions to maximize the value created by the Scrum Team

The Scrum Master serves the Product Owner in several ways, including:

• Finding techniques for effective Product Backlog management;
• Clearly communicating vision, goals, and Product Backlog items to the Development Team;
• Teaching the Scrum Team to create clear and concise Product Backlog items;
• Understanding long-term product planning in an empirical environment;
• Understanding and practicing agility; and,
• Facilitating Scrum events as requested or needed.

The Scrum Master serves the Development Team in several ways, including:

• Coaching the Development Team in self-organization and cross-functionality;
• Teaching and leading the Development Team to create high-value products;
• Removing impediments to the Development Team’s progress;
• Facilitating Scrum events as requested or needed; and,
• Coaching the Development Team in organizational environments in which Scrum is not yet fully adopted and understood.

The Sprint

The heart of Scrum is a Sprint, a time-box of one month or less during which a “Done”, useable, and potentially releasable product Increment is created. Sprints have consistent durations throughout a development effort. A new Sprint starts immediately after the conclusion of the previous Sprint. Sprints contain and consist of the Sprint Planning Meeting, Daily Scrums, the development work, the Sprint Review, and the Sprint Retrospective.

Daily Scrum

The Daily Scrum is a 15-minute time-boxed event for the Development Team to synchronize activities and create a plan for the next 24 hours. This is done by inspecting the work since the last Daily Scrum and forecasting the work that could be done before the next one. The Daily Scrum is held at the same time and place each day to reduce complexity. During the meeting, each Development Team member explains:

• What has been accomplished since the last meeting?
• What will be done before the next meeting?
• What obstacles are in the way?

The Development Team uses the Daily Scrum to assess progress toward the Sprint Goal and to assess how progress is trending toward completing the work in the Sprint Backlog

Product Backlog

The Product Backlog is an ordered list of everything that might be needed in the product and is the single source of requirements for any changes to be made to the product. The Product Owner is responsible for the Product Backlog, including its content, availability, and ordering.

A Product Backlog is never complete. The earliest development of it only lays out the initially known and best-understood requirements. The Product Backlog evolves as the product and the environment in which it will be used evolves. The Product Backlog lists all features, functions, requirements, enhancements, and fixes that constitute the changes to be made to the product in future releases. Product Backlog items have the attributes of a description, order, and estimate.

The Product Backlog is often ordered by value, risk, priority, and necessity. Top-ordered Product Backlog items drive immediate development activities. The higher the order, the more a Product Backlog item has been considered, and the more consensus exists regarding it and its value.

Sprint Backlog

The Sprint Backlog is the set of Product Backlog items selected for the Sprint plus a plan for delivering the product Increment and realizing the Sprint Goal. The Sprint Backlog is a forecast by the Development Team about what functionality will be in the next Increment and the work needed to deliver that functionality. The Sprint Backlog makes visible all of the work that the Development Team identifies as necessary to meet the Sprint Goal.

The Sprint Backlog is a plan with enough detail that changes in progress can be understood in the Daily Scrum. The Development Team modifies Sprint Backlog throughout the Sprint, and the Sprint Backlog emerges during the Sprint. This emergence occurs as the Development Team works through the plan and learns more about the work needed to achieve the Sprint Goal.

Extreme Programming Practices.

Extreme programming<ref>[3]</ref> is an agile software development methodology focussing on short incremental development incorporating constant feedback and adaptability. As opposed to the Scrum methodology that focusses on overall project development, extreme programming focusses on programming practices that improve the overall process efficiency and quality of the software product. Extreme programming has 12 practices, grouped into four areas, derived from the best practices of software engineering.

• Fine scale feedback
  • Pair programming : Two programmers work together at one workstation. One, the driver, types in code while the other, the observer, reviews each line of code as it is typed in. The two programmers switch roles frequently. The advantage of this technique, is that the expertise passes between the two programmers, the pair also keep each other honest with more likelihood of bugs being caught and unit tests being written as development progresses.
  • Planning game : The panning game is analogous to the Sprint planning in the Scrum process. The process involves gathering the requirements from the customer, prioritizing these and estimate the time required to implement these. This phase happens incrementally with the project being steered towards its eventual goal.
  • Test driven development : Extreme programming like other Agile methodologies emphases a test-as-you-go approach. Often, unit tests are written before the eventual code is coded. This approach is intended to stimulate the programmer to think about conditions in which his or her code could fail.
• Continuous process
  • Continuous integration : Programmers are encouraged to integrated into the main development tree on a very regular basis. This ensures that programmers are working on the latest version of the code and avoids integration issues down the road.
  • Small releases : The delivery of the software is done via frequent releases of live functionality creating concrete value. The small releases help the customer to gain confidence in the progress of the project.
• Shared understanding
  • Coding standard ; Coding standard is an agreed upon set of rules that the entire development team agree to adhere to throughout the project. The standard specifies a consistent style and format for source code, within the chosen programming language, as well as various programming constructs and patterns that should be avoided in order to reduce the probability of defects.
• System metaphor : The system metaphor allows all the stake holders to be able to explain the working of the system. It's a naming concept for classes and methods that should make it easy for a team member to guess the functionality of a particular class/method, from its name only
• Programmer welfare
  • Sustainable pace : This ensures that programmer welfare is part of the project development methodology. Programmers are not required to work more than a certain number of hours a week and are asked to work overtime if required only on alternating weeks.

Implementing Scrum

There are quite a few commercial and free tools available to implement the Scrum methodology. There are Scrum certification programs as well. We take a big picture view of what it takes to implement the Scrum methodology. Implementing Scrum can be done using the following steps:

1. Create the Scrum team
   1. The team should be cross-functional with representatives from all the stake holders
   2. There should be a willing product owner, who has management's support.
   3. There should be a Scrum master who drives the Scrum process.
2. Create and prioritize the product backlog :
   1. The Product Backlog, in its simplest form, is a list of things that people want to be done to the product, in priority order.
   2. The Product Backlog can contain anything related to the product, i.e. bugs, enhancements, whole projects, issues, risks etc.
   3. The Scrum process, and agile development principles generally, are collaborative and inclusive.
   4. Only the Product Owner can prioritize the Product Backlog, however one does need to explain the priority.
3. Sprint Planning/clarify requirements
   1. Have a Sprint Planning meeting that is attended by all the stakeholders.
   2. Decide on your Sprint duration, i.e., the period of each incremental development cycle.
   3. Select Target Backlog for Sprint using the Product backlog. The top items from the Product backlog should be worked on first.
   4. The Product owner is required to clarify the Sprint Requirements followed by a team discussion. This is to clarify the functional requirements of the task assigned to the Sprint.
   5. Estimate time required for the Sprint tasks.
4. Track the project progress
   1. Use daily Burndown Charts which are publicly displayed chart showing remaining work in the sprint backlog
   2. Update estimated time to complete on a daily basis.
   3. Annotate with key events
5. Review and Reflect
   1. At the end of the Sprint, hold a Sprint Review meeting to review what was delivered in the Sprint. Demo the software. Whether it’s complete, working software prior to a release, or work-in-progress in a long-running multi-Sprint project, demo what has been completed in the Sprint. Team members should demo the areas they have worked on.
   2. Following the review, reflect on various aspects of the completed Sprint and try to identify what worked well and what didn't.

Agile Tools

There are quite a few commercial as well as open-source<ref>[4]</ref> and free tools available to implement Agile methodologies in software development. Some of these are listed below:

• AgileFant
• Icescrum
• Agilo
• VersionOne
• TargetProcess.
• Rally

References

<references/>

External Links

Extreme Programming
Industrial XP
Scrum Tools
Agile Buddy Blog
DSDM