CSC/ECE 517 Fall 2007/wiki3 9 pp

Collective Ownership and Continuous Integration

Question:Two of the important Agile methodologies that we have not said much about in this course are collective ownership and continuous integration. What are the best definitions or examples you can find? Are these widely practiced? Are they controversial? Are there adherents of other design methodologies who consider them harmful? Investigate, through the Web and through the ACM DL.

What is Collective Ownership?

Collective code ownership means that everybody is responsible for all the code; this, in turn, means that everybody is allowed to change any part of the code. Pair programming contributes to this practice: by working in different pairs, all the programmers get to see all the parts of the code.

Advantages

• A major advantage claimed for collective ownership is that it speeds up the development process, because if an error occurs in the code any programmer may fix it.

• If you run across some code that could be improved, you have to stop and improve it, no matter what. The short term benefit is that your code is cleaner. The long term benefit is that the whole system gets better all the time, and everyone tends to be familiar with most of the system. Collective Code Ownership makes refactoring work better by exposing the team to more opportunities for big refactorings.

Disadvantages

• By giving every programmer the right to change the code, there is risk of errors being introduced by programmers who think they know what they are doing, but do not foresee certain dependencies

Continuous Integration

What is Continuous Integration

Continuous Integration is a powerful agile development practice where integration and build is performed continuously, or as frequently as possible. It encourages developers to integrate their code often to mainstream so that unexpected conflicts do not go undetected for longer periods of time. This concept has originated from extreme programming as one of its twelve original practices and has emerged as a core concept in agile development.

Need for continuous integration

In traditional code development environment, whenever programmer needs to make changes to the system, s/he checks out the relevant files from central version control system, works independently on his checked out version of the code and before submission, runs the new code with latest build available.

However, though s/he may be able to build and run it correctly on his/her own system, some other conflicts may go undetected until central repository code undergoes a build. Such central repository builds are typically once a week in traditional development scenario. So failures persist in system for longer periods of time and are harder to detect and fix. Even if there are no errors, integrated code is available only after a week for release.

One solution to the problem is to have daily builds so that conflicts and failures are detected early and hence can be fixed as soon as they are detected. This maintains the sanity of central repository code. Typically, iterative development projects that make use of prototypes benefit from this approach as daily releases are possible.

Continuous integration is a powerful technique that extends this idea further by having developers integrate and release code into the central code repository every few hours, whenever possible. Only one person or one pair of programmers (in case of pair programming) integrates at a time so that compatibility problems are reduced to a minimum. This solves the problem of “Integration hell” **link_to http://c2.com/cgi/wiki?IntegrationHell where there are lot of problems integrating code from different developers.

Figure indicating continuous integration process: Source: http://www.think-box.co.uk/blog/2006/02/ten-minute-build-continuous.html

Definitions of Continuous Integration

There are a lot of definitions, of continuous integration available on the web, that help to understand the concept clearly. We have attempted to mention below the most significant ones below:

Definition: “Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”

Source: http://www.martinfowler.com/articles/continuousIntegration.html Author: Martin Fowler, renowned author, speaker on software architecture and one of the leaders of agile development movement.

Comment: This is a much quoted, comprehensive definition that introduces continuous integration and explains its advantage over other approaches.

Definition:

“Continuous Integration (CI) is the practice of periodically executing Automated Build Scripts (ABS), preferably in short intervals. There are two flavors of this practice. The first flavor, mostly applied by early XP practitioners states that the system is built by one developer at any given time. In this flavor of CI Automated Build Scripts are still required, but kicking off the build happens manually. Since this version relies heavily on the entire team to be co-located, it does not scale well in distributed agile projects.

The second flavor of CI relies on a computer background process to poll the code repository in short intervals and automatically trigger the build process if it detects any changes since the last build. In this version of CI, the developer is only responsible to build locally and check-in the code. This model is better suited for highly scalable agile teams because of its reliance on automated asynchronous processes to trigger the build. This gives distributed teams an early feedback on the status of the system and therefore it enables dislocated team members and managers to have the same quick feedback as the team itself.”

Source: http://www.jacoozi.com/blog/?p=18

Comment: This definition explains the ways in which continuous integration can be implemented using automated build scripts to achieve frequent integration.

Definition: “The macro process of object-oriented development is one of "continuous integration." ... At regular intervals, the process of "continuous integration" yields executable releases that grow in functionality at every release. ... It is through these milestones that management can measure progress and quality, and hence anticipate, identify, and then actively attach risks on an ongoing basis.”

Source: Grady Booch, Object-Oriented Analysis and Design with Applications, 2nd edition.

Comment: This definition was introduced by Grady Booch in the context of Object Oriented analysis and design. It illustrates that Continuous integration is not limited to tools that achieve it but rather it is one of the best software development practices.

Definition: “Continuous integration allows that team to measure their progress in compile-able working code. It gives the team instant feedback on the health of the build and the state of code repository. Setting up continuous integration also sets the team up for reaping the value of automated testing once they grow enough to include that in the agile arsenal.”

Source: http://www.commandinformation.com/blog/?p=30 Comment: This definition points out how continuous integration can act as a means to achieve fast positive feedback cycles during software development.

Definition: “Continuous integration basically means that all changes to the software are integrated with the rest of the software as soon as the developer submits the change to the central repository. At that point, or at very near intervals, the whole system is run through unit and integration tests to see that it is still healthy. If a defect arises, either through a mistaken submission, or the submission of something that breaks something else, the system alerts the developer, and possibly relevant management.”

Source: http://www.agileadvice.com/archives/2005/07/why_should_busi.html

Comment: This definition describes the how continuous integration works in response to changes to software.

Definition:

“Continuous integration is

• A central repository for all members of a team, containing:
  • the latest code (at least)
  • the latest executables
• An automated process for building and testing all project assets that:
  • can be run many times a day
  • is self sufficient

It is best to think of continuous integration as a mindset or statement of intent that allows you to reduce risk by frequently integrating incremental software development changes. Basically, it represents the realization and refinement of a common software development best practice: the daily build and smoke test.”

Source: http://www.ibm.com/developerworks/rational/library/sep05/lee/ Author: Kevin A. Lee, Technical Consultant, IBM

Comment: Continuous integration defined in terms of resources needed for it.

Examples

Real-world scenario: Building small feature using Continuous integration

Reference: http://martinfowler.com/articles/continuousIntegration.html#BuildingAFeatureWithContinuousIntegration Martin Fowler explains what CI is and how it works by giving an example. It shows the way it is used with the development of a small feature. The example is long but comprehensive in a way that illustrates use of continuous integration in a real-world scenario. The excerpt below is adapted from reference site mentioned above.

Scenario:

Developer begins by taking a copy of the current integrated source onto his/her local development machine. S/he does this by using a source code management system by checking out a working copy from the mainline.

Now s/he takes modifies working copy, adding or changing code and automated tests. Continuous Integration assumes a high degree of tests which are automated into the software. Once done s/he carries out an automated build on his/her development machine. This takes the source code in his/her working copy, compiles and links it into an executable, and runs the automated tests. Only if it all builds and tests without errors then the overall build considered to be good.

With a good build, developer can then think about committing changes into the repository. However, other people may, and usually have, made changes to the mainline before s/he gets chance to commit. So first s/he updates his/her working copy with their changes and rebuild. If their changes clash with her/his changes, it is developer’s responsibility to fix errors if any and repeat until s/he can build a working copy that is properly synchronized with the mainline. Once s/he has made his/her own build of a properly synchronized working copy s/he can then finally commit changes into the mainline, which then updates the repository. However this commit doesn't finish his/her work. At this point s/he builds again, but this time on an integration machine based on the mainline code. Only when committed changes build successfully on the integration is his/her job done. This integration build can be executed manually or could be automated.

If a clash occurs between two developers, it is usually caught when the second developer to commit builds their updated working copy. If not the integration build should fail. Either way the error is detected rapidly. At this point the most important task is to fix it, and get the build working properly again. In a Continuous Integration environment you should never have a failed integration build stay failed for long. A good team should have many correct builds a day. Bad builds do occur from time to time, but should be quickly fixed. The result of doing this is that there is a stable piece of software that works properly and contains few bugs. Everybody develops off that shared stable base and never gets so far away from that base that it takes very long to integrate back with it. Less time is spent trying to find bugs because they show up quickly.

Use of tools in implementation of Continuous integration:

Example tool: CruiseControl

References: http://www.ibm.com/developerworks/rational/library/sep05/lee/ http://cruisecontrol.sourceforge.net/index.html The figure below shows infrastructure for a typical CruiseControl implementation

The process that CruiseControl follows is: To monitor the SCM repository for changes; using Rational ClearCase (version control system), it looks for check-ins performed on a branch or stream (usually the project's integration stream). If changes are found, then, on a schedule, CruiseControl builds the application via existing Ant or Maven scripts, runs JUnit Test suite, and reports on the results of build. It can send these results automatically to a set of users via email and/or hold them centrally on a build-results Web site. Optionally, CruiseControl can also deploy an application and carry out any additional scripted task as required

Video presentation on Continuous integration implementation

Source: http://www.integratebutton.com/

In this video, Paul Duvall walks through an example of integrating button using continuous integration technology.

Penetration of Continuous Integration in Industry

According to 2nd Annual “The state of Agile Development” Survey was conducted by VersionOne Inc. and sponsored by APLN, results compiled:

Survey statistics:

Organizations that ---

• Currently use continuous integration tools: 57%
• Plan to use continuous integration tools: 17%
• Would like to use continuous integration tools: 20%
• Don’t need continuous integration tools: 6%

50% of organizations already employ continuous integration in agile practices.

This global survey was held in June and July of 2007 and was completed by almost 1700 people from 71 countries.

Link: http://www.versionone.com/pdf/StateOfAgileDevelopmet2_FullDataReport.pdf

This shows that the continuous integration concept is picking up well in the industry.

Controversy over Continuous integration

Opponents’ claims on continuous integration:

Reference: http://c2.com/cgi/wiki?ContinuousIntegration

“Opponent claim: What if you release from your machine and I release from mine and we step on each other's changes? Proponent argument: It hurts when you do that. Don't do that. Consider having a SingleReleasePoint. Go to it; load your changes, test, and release. This makes releasing single-threaded, which makes it quite simple to detect conflicts. Single release machines are good. Although we allow developers to version and release code that they have tested, we don't update a config map and create a package until the code has been tested on a single machine, integrated with all of the other changes. That way, if all is not well, we can recover by reloading the previous map. The new map is created on the same single machine”. -- Matt Stephenson