CSC/ECE 517 Fall 2011/ch6 6d sk

6d. The Agile landscape. What are the main agile development methodologies? What are their origins? How do they differ? How widely are they practiced? What evidence exists for their effectiveness?

---

The Agile Landscape

Overview

Traditionally, software has been developed using rigid methodologies such as the Waterfall method of software development. The Waterfall methodology is a set of stages such as - Analysis, Design, Implementation, Testing and Maintenance. These stages follow a strictly linear sequence as mentioned in the last sentence. This method works perfectly well when a project is completely planned, but it has been observed that it is not always the case. The waterfall model is not capable to handle changes requested by the customers when the project has gone deep into the pipeline. In such cases, all the processes have to be followed to incorporate the requested changes, which consumes time and costs dearly to the customer.

Developers who were facing this problem more often started to realize that real world tends to have quite an impact on the development of any process or product. It was time to have a new strategy for software development which was capable enough to overcome the shortcomings of traditional software development methodologies. There are various such methodologies which provide fast and iterative development, ability to handle late changes in requirements and many more features. These methodologies are collectively called as Agile Software Development Methodologies.

The following article provides a brief introduction to Agile Software Development, the various methodologies for agile development, their comparison and some statistics to demonstrate the acceptance and the success rates of agile methodologies.

Introduction to Agile Software Development

In 1990's, software developers started to realize that a planned waterfall approach for software development had many shortcomings and did not facilitate speedy and flexible software development. There were many efforts to devise new methodologies which overcame those shortcomings. These methodologies were called the Agile Methodologies for software development. They allowed developers to adapt to the changes in demands during software development as well as incorporate future changes to the software during any stage of software development.

The agile methodologies are based on iterative and incremental development. In organizations who follow agile development, requirements for a product are formed mainly through collaboration between small teams which are cross-functional and self-organizing. It allows the process of software development to be iterative, time bound, adaptive, evolutionary and flexible. A team learns how to be Agile in the environment provided by the Agile processes. There is more importance to how a team works together, allowing teams to work in a cohesive fashion, increasing the productivity many times. The increase in productivity is such as a team can provide a working software in the first week, and then improve with subsequent iterations.

Agile Manifesto

n February 2001, a group of software developers, interested in developing new lightweight methodologies for software development, published the Manifesto for Agile Software Development. The Agile Manifesto forms the core of agile software development. The manifesto reads as follows: [1]

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The text in bold face are the main terms used in the Manifesto. We describe each of them below: [2]

Individuals and Interactions: Agile methodologies are focused on individuals and interactions between them. They have more emphasis on self-organization and motivation and are based on the principles of co-location and pair-programming.

Working Software: A working software developed quickly will be useful rather than presenting documents to customers

Customer Collaboration: It is not possible to fully gather requirements at the beginning of a software development cycle. so it is important to keep the customer involved during the development process and receive their feedback

Responding to change: Quickly respond to changes in requirements through continuous iterative development process.

Principles of Agile Manifesto

The supporters of Agile Manifesto follow the 12 principles mentioned below:[3]

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

• Business people and developers must work together daily throughout the project.

• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

• Working software is the primary measure of progress.

• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

• Continuous attention to technical excellence and good design enhances agility.

• Simplicity--the art of maximizing the amount of work not done--is essential.

• The best architectures, requirements, and designs emerge from self-organizing teams.

• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

If you believe in and endorse the Agile Manifesto, you can become a signatory here.

Agile Methodologies

Extreme Programming

XP Process Model

Extreme programming (XP) is a popular approach to agile software development and was created by Kent Beck in 1996. XP is defined by a set of five values that establish a foundation for all work performed as part of development process. These five values are communication, simplicity, feedback, courage, and respect. Each individual value serves as a driver for specific activities, actions, and tasks.

• Communication - XP emphasize close, yet informal communication between customers and developers. This helps ensure effective communication between software engineers and other stakeholders.
• Simplicity - In order to achieve simplicity, XP requires developers to design only for the immediate needs of the project. This results in a simple design that can be implemented easy.
• Feedback - Feedback is received from three sources: software tests, the customer, and other software team members.
• Courage - In order to adhere to certain XP practices, Beck states that is requires courage.
• Respect - By following the other four values, an agile team will develop respect among its members and other stakeholder. As the team begins to achieve success in their delivery of software increments, they will also develop respect for the XP process itself.

XP consists of four framework activities: planning, design, coding, and testing as seen in the figure to the right. Each phase is summarized in the paragraphs that follow.

• Planning - The initial phase of the process involves the planning game. It is a requirements gathering activity that enables the members of the team to understand the business context for the software and involves listening to the customer. This leads to the creation of a set of user stories that describe the required output, features, and functionality of the software. Each story written by the customer is written on a index card and assigned a value based on its overall business value. Members of the team will then determine a cost (measured in weeks) for each story based on their estimation for how much work would be required to implement the story. The customers and developers will then work together to determine which group of stories will be implemented in the next iteration of the software.
• Design - The design phase of the process involves the creation of a simple design of the system and also provides implementation guidance for each story to be included in the iteration.
• Coding - After a preliminary design for the system is created, the team will then develop a set of unit tests for the system. These tests will serve as guidelines for the developers to focus only on getting the tests to pass and nothing more. One of the defining aspects of XP, involves the pair programming concept. XP recommends that two programmers should work together at one computer workstation to develop code. One person writes the code, while the other will watch and provide feedback. After a period of time, the two programmers will switch their roles.
• Test - In order to place an emphasize on tests, the unit tests generated during the code phase are ran anytime the code is modified to ensure no problems exist in the codebase.

Scrum

Scrum Process Model

Scrum (derived from a term used in rugby) is an agile process created by Jeff Sutherland in the early 1990s. The basic principles of Scrum are consistent with the Agile Manifesto and serve to guide development. Some of the distinguishing features of Scrum include:

• Work occurs in “sprints” and is derived from a “backlog” of existing requirements
• Testing and documentation are on-going as the product is constructed
• Meetings are very short and sometimes conducted without chairs
• Product demonstrations are delivered to the customer with the time-box allocated

The overall flow of the Scrum process is illustrated in the diagram to the right. Scrum emphasizes the use of a set of software process patterns that have proven effective for projects with tight timelines, changing requirements, and business criticality.

The defining characteristics of Scrum include:

• Product Backlog - A list of project features that provide business value for the customer. New requirements can be added to the backlog at any time. The project manager is responsible for accessing the backlog and updating the priority of each individual requirement.
• Sprint Backlog - Each iteration of the process (called a Sprint) involves choosing a subset of the product backlog that will be implemented.
• Sprint - A typical sprint occurs over a 30 day time-box and involves the implementation of requirements currently in the sprint backlog.
• Scrum Meetings - Every 24 hours, the project team will hold a meeting to discuss three key questions; what did you do since the last team meeting?, what obstacles are you encountering?, what do you plan to accomplish by the next team meeting?
• Release - At the end of each sprint, a working iteration of the software is release to the customer to evaluate. Any new features requested will then be entered into the product backlog for another iteration of the software.

Dynamic Systems Development Method

DSDM Process Model

The Dynamic Systems Development Method (DSDM) is another agile software process that "provides a framework for building and maintaining systems which meet tight time constraints through the use of incremental prototyping in a controlled project environment." DRDM is an iterative process which follows the 80 percent rule, meaning that 80 percent of an application can be delivered in 20 percent of the time it would take to deliver the complete application.

DSDM is structured into five phases:

• Feasibility study - The initial feasibility study establishes the business requirements and any constraints associated with the application. This phase is also used to determine if the application is a viable candidate for the DSDM process.

• Business Study - In this phase, the functional and information requirements are established which gives the application business value. The basic architecture of the application and maintainability requirements for the application are also defined.

• Functional Model Iteration - In the fuctional model interation phase, a set of incremental prototypes of the application are demonstrated to the customer to show functionality. By gathering feedback from the customer, the team can gather additional requirements to be included as part of the application.

• Design and Build Iteration - The project team will revisit the prototypes built during the functional model iteration to ensure that they have been designed in a way that they will provide operational business value to the customer. This phase can run concurrently with the functional model iteration phase in some cases.

• Implementation - In the implementation phase, the lastest software increment is deployed to the customer.

Crystal Family of Methodologies

Choosing a Crystal Methodology

The Crystal family of agile methods were create by Alistair Cockburn and Jim Highsmith. The main goal was to achieve a software development approach that puts an emphasize on "maneuverability" during "a resource-limited, cooperative game of invention and communication, with a primary goal of delivering useful, working software and a secondary goal of setting up for the next game."

A defining part of Crystal is the use of several different colors to indicate the "heaviness" of the process. A project team trying to determine which process to pick would consider its size and criticality as the major factors. In addition to multiple colors, there are four levels of potential loss as a result of a system failure.

These four levels from smallest to largest potential loss are:

• Comfort (C)
• Discretionary Money (D)
• Essential Money (E)
• Life (L)

The lowest potential loss would be Comfort, in this case a system failure would simply be an inconvenience for the customer. On the other end of the spectrum, Life as the name implies means that a system failure would result in loss of life.

Crystal Clear & Orange

Two of the more well known methodologies in the Crystal family are Crystal Clear & Orange. The only difference between the two is that Crystal Clear is designed for one project team while Crystal Orange is for a multi-team project.

The fundamental principles of both include:

• Incremental delivery
• Project tracking
• End-user participation
• Automated Regression
• Two user viewing
• Workshop

Feature Driven Development

FDD Process Model

Feature-Driven Development, or FDD, is an agile process that is model-driven. Models, which are written in UML and other standardized languages, provide a way to identify units of functionality and understanding the context in which the units are used. It is entirely focused around discovering and implementing system features. A system feature is any distinct unit of functionality that is valued by the client. In most cases, features correspond to system use cases, but may also involve functionality not directly used by an external actor (for example the customer).

The FDD process involves five phases:

• Develop an Overall Model
• Build a Features List
• Plan by Feature
• Design by Feature
• Build by Feature

The first three phases, which focus on planning, are performed at the beginning of a project, while the final two phases are typically implemented by a series of short iterations. Other then the software deliverable, the only documentation includes system feature and class models.

Adaptive Software Development

ASD Process Model

Adaptive Software Development (ASD) was proposed by Jim Highsmith as an agile process model for building complex software and systems. The two core ideologies of the process are human collaboration and team self-organization.

The three phases of ASD are:

• Speculation - In this initial phase, the project is started by conducting adaptive cycle planning. Adaptive cycle planning uses the customer mission statement, project constraints, and basic requirements to define the set of release cycles (software increments) that wil be part of the project. Since the cycle plan will inevitably change, after completion of the first cycle, it is reviewed and adjusted to better fit the reality in which the team is working.

• Collaboration - Next in collaboration, the project team performs requirements gathering and performs joint application development (JAD). A JAD session involves developers and customer representatives meeting to discuss project features and serves as a way to enhance communication.

• Learning - The final learning phase is used to conduct quality assurance and deployment of the application. Lessons learned during the entire ASD process is then incorporated into the next iteration of the software.

Agile Unified Process

AUP Process Mode (Disciplines performed are shown on Y Axis)

The Agile Unified Process (AUP) is a simplified version of the Rational Unified Process (RUP) created by IBM. It provides an easy to understand approach to develop applications using the defining agile concepts while still remaining true to RUP.

The four phases of RUP are:

• Inception - The goal of this phase is to determine the overall scope of the project, as well as obtaining project funding and determining a potential architecture of the application.
• Elaboration- In the elaboration phase, the architecture proposed during the previous phase is expanded and developed into a usable design.
• Construction- In the construction phase, the goal is to build working software on an incremental basis while ensuring the highest-priority requirements are being implemented first.
• Transition- Lastly, the transition phase is used for validation and project deployment.

Other Methodologies

Other less known agile methodologies include:

• Agile Modeling
• Lean Software Development
• Essential Unified Process
• Open Unified Process
• Velocity Tracking

Comparison of different agile methodologies

In this section, we compare the different agile methodologies mentioned in this text based on multiple aspects. In general, the agile life cycle is more or less similar in all the methodologies. It involves phases such as - Project Initiation, Planning, Requirement Elaboration, Architecture design followed by one or more releases. Each release is made up of timeboxes which are 1 -6 weeks long and have a fixed delivery date. [4][5]

Generic Agile Life Cycle

Project Initiation

Generic Agile Life Cycle

This stage involves setting up project and justifying the need and the business case for the project. It also involves elaborating vary high level requirements. Not all methodologies give equal importance for justifying the business case for the project, and have different names for the vision statement that mentions the goal of the project. The level of formality varies greatly from one method to another. It is important to know the practices followed by each method in order to select the right method for the project.

Formality and Vision

Scrum is the least formal of all. Requirements are documented as Backlogs and the only other documentation is the code. The "sprint Goal" is the timebox vision statement for Scrum.

In XP, the documentation involves Stories, Running code and the Tests. XP is a little more formal than Scrum. The "Metaphor" is the product vision for XP.

Crystal Orange is much more formal and has documentation in the form of Release sequences, Project Schedule, Requirements document, Use case documents, Design sketches, Object models, Running code, Test cases, User manuals, Status reports and Inter-team specifications.

DSDM also provides a high level of formality. It has documents such as Feasibility report which is the project vision, Outline plan, Business Area Definition, Requirements List, System Architecture Definition, Development Plan, Functional Model, Functional Prototype, Design Prototype, Tested system, Delivered system, Implementation Plan, Development Risk Analysis Report, Review Records, Test records, User documentation and Project Review Document.

FDD does not cover all the stages of the life cycle, but still provides feature lists and running code.

ASD and AUD too provide documentation in the form of Requirements, Running code, Test cases, etc.

Requirements Gathering

In Scrum, a Product Backlog List is created which is maintained by the Product Owner.

In XP, requirements are mentioned in the form of Stories by the customer. Projects which use XP generally have a collection of such Story cards. Scrum and XP do not involve a separate stage for Elaborating Requirements.

In FDD, a feature list is created from an existing requirements document.

In Crystal Orange and DSDM have a separate Elaboration phase in workshops are conducted to identify high level requirements which are documented in a Requirments document.

ASD provides Joint Application Development (JAD) sessions where the customers and developers discuss the features to be included in the product.

Project Planning

Having a project plan is always beneficial to track the project and take to completion in desirable time. The methods mentioned in this article have different rules for preparing a project plan. Some methods such as DSDM make it mandatory to have a project plan while others do not give much emphasis on having one. All the agile methods have a similar approach toward project planning. Releases and Timeboxes are basic elements of a project plan in all the methods. The plan includes multiple Releases with ussually 3 Timeboxes in each Release, except for DSDM which has only one Release.

Estimating the time required for to develop a particular component or a feature is done by different individuals