CSC/ECE 517 Fall 2014/ch1a 16 va: Difference between revisions
No edit summary |
|||
| (15 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
'''<big>Continuous Integration Pipelines</big>''' | '''<big>Continuous Integration Pipelines</big>''' | ||
[https://docs.google.com/document/d/1eq3XHiUUBIrEBx-R2Mgolaw7HttSgzPniwDLEF5oVk0/edit?pli=1# Writeup for this topic] | |||
__TOC__ | __TOC__ | ||
== Introduction == | == Introduction == | ||
[http://en.wikipedia.org/wiki/Continuous_integration Continuous Integration] Pipeline is a [http://en.wikipedia.org/wiki/Software_development software development] technique, which focuses on automating major portions of the build, test and deployment tasks. In CI pipelining, any change made by the developer to their projects’ shared source code repository automatically triggers a build. Later on, when the build and automated [http://en.wikipedia.org/wiki/Unit_testing unit-tests] are successful, the application is automatically deployed to a production-like environment where more rigorous testing can be done. <ref>[http://www.asp.net/aspnet/overview/developing-apps-with-windows-azure/building-real-world-cloud-apps-with-windows-azure/continuous-integration-and-continuous-delivery Continuous Integration and Continuous Delivery (Building Real-World Cloud Apps with Azure)] by By Tom Dykstra, Rick Anderson and Mike Wasson | June | [http://en.wikipedia.org/wiki/Continuous_integration Continuous Integration] Pipeline is a [http://en.wikipedia.org/wiki/Software_development software development] technique, which focuses on automating major portions of the build, test and deployment tasks. In CI pipelining, any change made by the developer to their projects’ shared source code repository automatically triggers a build. Later on, when the build and automated [http://en.wikipedia.org/wiki/Unit_testing unit-tests] are successful, the application is automatically deployed to a production-like environment where more rigorous testing can be done. <ref>[http://www.asp.net/aspnet/overview/developing-apps-with-windows-azure/building-real-world-cloud-apps-with-windows-azure/continuous-integration-and-continuous-delivery Continuous Integration and Continuous Delivery (Building Real-World Cloud Apps with Azure)] by By Tom Dykstra, Rick Anderson and Mike Wasson | 12 June, 2014</ref>.This technique allows the contributors of the project to build, test and deploy their code, easily and quickly, that too with greatly reduced risk. Thus, the CI pipelining ultimately ensures the rapid delivery of a quality software product.<br> | ||
== | == The Traditional Software Development Workflow == | ||
Consider the following workflow that demonstrates the development to deployment phase in a traditional [http://en.wikipedia.org/wiki/Software_development_process software development lifecycle:]<br> | Consider the following workflow that demonstrates the development to deployment phase in a traditional [http://en.wikipedia.org/wiki/Software_development_process software development lifecycle:]<br> | ||
<br> | <br> | ||
| Line 23: | Line 24: | ||
== Working of a Continuous Integration Pipeline == | == Working of a Continuous Integration Pipeline == | ||
The typical workflow of the development to deployment phase in a CI pipeline is as follows: <ref name="Martin">[http://www.martinfowler.com/articles/continuousIntegration.html An Article on Continuous Integration] by Martin Fowler | 1 May, 2006</ref><ref name="ThoughtWorks">[http://www.thoughtworks.com/continuous-integration ThoughtWorks-Continuous Integration]</ref><br> | The typical workflow of the development to deployment phase in a CI pipeline is as follows: <ref name="Martin">[http://www.martinfowler.com/articles/continuousIntegration.html An Article on Continuous Integration] by Martin Fowler | 1 May, 2006</ref><ref name="ThoughtWorks">[http://www.thoughtworks.com/continuous-integration ThoughtWorks-Continuous Integration]</ref><br> | ||
<br> | <br> | ||
| Line 34: | Line 32: | ||
1. Developers | 1. The Developers, who are a part of the Delivery team, will take the current code base from a shared code repository, which is maintained for their project in a [http://en.wikipedia.org/wiki/Revision_control version control] system.<br> | ||
2. The Developers will then write their own code on top of the current code base taken from the shared code repository. <br> | |||
3. When the changes are done, the developers integrate their code with the shared code repository. This should happen frequently (at least once a day).<br> | |||
4. An [http://en.wikipedia.org/wiki/Build_automation automated build] is run, every time a developer integrates his changes. [http://en.wikipedia.org/wiki/Unit_testing Unit tests] and [http://en.wikipedia.org/wiki/Integration_testing Integration tests] are also performed in this phase. If either of the build or the tests fail, the development team is notified to fix the errors. The development team is also notified on a successful build.<br> | |||
5. Once the build and the above tests succeed, the software is automatically deployed to a testing environment where more rigorous testing can be done.<br> | |||
6. If the in-depth testing succeeds, the software product with the new functionality becomes release-ready. If the testing fails, the development team is notified to fix the errors. The development team is also notified if the tests are successful.<br> | |||
7. As soon as the customer requires the new functionality in the software, the release-ready build is deployed into the production environment.<br> | |||
== Advantages of Continuous Integration Pipelines == | == Advantages of Continuous Integration Pipelines == | ||
By implementing CI pipelining in software | CI pipelining significantly reduces the risks posed by taking the traditional approach during the development to deployment phase of a software. By implementing CI pipelining in software projects the following advantages are achieved:<ref name="Martin"></ref><ref name="ThoughtWorks"></ref><ref>[http://en.wikipedia.org/wiki/Multi-stage_continuous_integration Multi-Stage Continuous Integration]</ref><br> | ||
<ul> | <ul> | ||
<li>If any bugs are present in the code, they are detected early. Hence, the effort needed in removing the bug reduces significantly, since a comparatively smaller part of the system is affected and the concerned developers have recently worked on the code, so it is fresh in their minds. </li> | <li>If any bugs are present in the code, they are detected early. Hence, the effort needed in removing the bug reduces significantly, since a comparatively smaller part of the system is affected and the concerned developers have recently worked on the code, so it is fresh in their minds. </li> | ||
| Line 74: | Line 73: | ||
<br> | <br> | ||
=== Apache Continuum === | === Apache Continuum === | ||
[[File:ApacheContinium.png|right|frame|Apache Continuum's Build Status Screen <ref>[http://continuum.apache.org/docs/1.4.2/user_guides/report/index.html Apache Continuum - Project Builds Report | 5 June, 2014 | [[File:ApacheContinium.png|right|frame|Apache Continuum's Build Status Screen <ref>[http://continuum.apache.org/docs/1.4.2/user_guides/report/index.html Apache Continuum - Project Builds Report] | 5 June, 2014</ref>.The figure shows visual feedback provided to the user with the help of color Green (Indicating Build Success)]] | ||
Apache Continuum is an enterprise-ready continuous integration server. It has a web-based user-interface that helps in monitoring the current status of a build. Continuum comes as a standalone application and also as a | Apache Continuum is an enterprise-ready continuous integration server. It has a web-based user-interface that helps in monitoring the current status of a build. Continuum comes as a standalone application and also as a [http://en.wikipedia.org/wiki/WAR_(file_format) WAR] file. <ref name="cont">[http://continuum.apache.org Apache Continuum Tool]/</ref><ref>[http://www.avajava.com/tutorials/lessons/what-is-continuum-and-how-do-i-install-it.html What is Continuum and how do i install it?] by Deron Eriksson</ref><br> | ||
<br> | <br> | ||
Advantages: <ref name="compare"></ref><ref name="ibm">[http://www.ibm.com/developerworks/library/j-ap09056/ Choosing a Continuous Integration Server] by Paul Duvall | 5 September, 2006</ref><br> | Advantages: <ref name="compare"></ref><ref name="ibm">[http://www.ibm.com/developerworks/library/j-ap09056/ Choosing a Continuous Integration Server] by Paul Duvall | 5 September, 2006</ref><br> | ||
| Line 84: | Line 83: | ||
<li>The learning curve for Continuum is low. </li> | <li>The learning curve for Continuum is low. </li> | ||
</ul> | </ul> | ||
Disadvantages: <ref>[http://www.thinkplexx.com/learn/article/build-chain/ci/choosing-continuous-integrationci-tool-opensource-vs-professional-cruisecontrol-hudson-continuum-vs-teamcity CI Tool Comparison] April | Disadvantages: <ref>[http://www.thinkplexx.com/learn/article/build-chain/ci/choosing-continuous-integrationci-tool-opensource-vs-professional-cruisecontrol-hudson-continuum-vs-teamcity CI Tool Comparison] 2 April, 2010 </ref> | ||
<ul> | <ul> | ||
<li>Continuum does not have a very wide variety of plug-ins and extensions.</li> | <li>Continuum does not have a very wide variety of plug-ins and extensions.</li> | ||
| Line 109: | Line 108: | ||
=== Go === | === Go === | ||
[[File:Go.png|right|frame|A Go Pipeline <ref>[http://marcovaltas.com/img/posts/blog_pipeline.png A Go Pipeline]</ref>]] | [[File:Go.png|right|frame|A Go Pipeline <ref>[http://marcovaltas.com/img/posts/blog_pipeline.png A Go Pipeline]</ref>.The figure shows visual feedback provided to the user with the help of colors Red (Indicating Build Failure) and Green (Indicating Build Success).]] | ||
Go is an open source tool, built by ThoughtWorks Studios, that is used to support continuous delivery. It helps create and manage automated deployment pipelines. It supports automating the entire build-test-release process from check-in to deployment.<ref>[http://martinfowler.com/articles/go-interview.html Open-Sourcing ThoughtWorks Go] by Martin Fowler | 25 March, 2014</ref><br> | Go is an open source tool, built by ThoughtWorks Studios, that is used to support continuous delivery. It helps create and manage automated deployment pipelines. It supports automating the entire build-test-release process from check-in to deployment.<ref>[http://martinfowler.com/articles/go-interview.html Open-Sourcing ThoughtWorks Go] by Martin Fowler | 25 March, 2014</ref><br> | ||
<br> | <br> | ||
| Line 127: | Line 126: | ||
<br> | <br> | ||
<br> | <br> | ||
=== Hudson === | |||
[[File:Hudson.png|right|frame|Hudson's Build Status Screen <ref>[https://jazz.net/library/content/articles/rtc/2.0/build-hudson/ Using the Hudson build integration system with Rational Team Concert] by Jim D'Anjou, Jazz Jumpstart Team, IBM | October | === Hudson === | ||
[[File:Hudson.png|right|frame|Hudson's Build Status Screen <ref>[https://jazz.net/library/content/articles/rtc/2.0/build-hudson/ Using the Hudson build integration system with Rational Team Concert] by Jim D'Anjou, Jazz Jumpstart Team, IBM | 21 October, 2009</ref>. The build summary screen shows that a build failure has occured and the reasons for the failure.]] | |||
Hudson is a free continuous integration tool written in Java. It supports various SCM tools including CVS, Subversion, Git, Perforce, Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. Hudson became a popular alternative to CruiseControl and other open-source build servers in 2008.<ref>[http://en.wikipedia.org/wiki/Hudson_(software) Hudson (Wikipedia)]</ref> | Hudson is a free continuous integration tool written in Java. It supports various SCM tools including CVS, Subversion, Git, Perforce, Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. Hudson became a popular alternative to CruiseControl and other open-source build servers in 2008.<ref>[http://en.wikipedia.org/wiki/Hudson_(software) Hudson (Wikipedia)]</ref> | ||
| Line 150: | Line 150: | ||
<br> | <br> | ||
<br> | <br> | ||
== Challenges in a Continuous Integration Pipeline Implementation == | |||
Some of the challenges involved in implementing a CI pipeline are as follows:<ref>[https://www.ibm.com/developerworks/community/blogs/c914709e-8097-4537-92ef-8982fc416138/entry/continuous_integration_ci_practice_and_challenges_part12?lang=en CI Practices and Challenges] by Cherifa Mansoura | 10 March 2013</ref> | |||
<ul> | |||
<li>The infrastructure (hardware for the build machines, a version control system etc.) required for implementing a continuous integration pipeline may be very costly, especially for small software projects.</li> | |||
<li>Some practices involved in CI, like integrating code frequently with the shared code repository, require a lot of discipline on the part of the developers. It might not always be possible to maintain this discipline especially in larger Projects, which may involve a large number of developers working together on many components of the software. </li> | |||
</ul> | |||
== See Also == | == See Also == | ||
Latest revision as of 18:34, 25 September 2014
Continuous Integration Pipelines
Introduction
Continuous Integration Pipeline is a software development technique, which focuses on automating major portions of the build, test and deployment tasks. In CI pipelining, any change made by the developer to their projects’ shared source code repository automatically triggers a build. Later on, when the build and automated unit-tests are successful, the application is automatically deployed to a production-like environment where more rigorous testing can be done. <ref>Continuous Integration and Continuous Delivery (Building Real-World Cloud Apps with Azure) by By Tom Dykstra, Rick Anderson and Mike Wasson | 12 June, 2014</ref>.This technique allows the contributors of the project to build, test and deploy their code, easily and quickly, that too with greatly reduced risk. Thus, the CI pipelining ultimately ensures the rapid delivery of a quality software product.
The Traditional Software Development Workflow
Consider the following workflow that demonstrates the development to deployment phase in a traditional software development lifecycle:
1. Initially, the developers will code their individual modules.
2. Once the development is completed, the various modules are integrated together.
3. Once the merge is completed, the entire product is deployed in the testing environment.
4. Once the testing is completed, the product becomes release ready and is later on deployed in the production environment.
Generally, the above workflow is very risky to follow, since many severe issues may arise in above mentioned steps. These are stated below:
- Longer Integration Times: The code written by different developers might not integrate successfully with each other, resulting in build failures. Since, many code modules are integrated in one go, it is very difficult and time consuming to detect and resolve the cause of the build failure.
- Longer Testing Times: Once the complete product is ready, it may take a lot of time to perform the complete end-to-end testing of the software. Also, if a bug arises in the code, it may be difficult to resolve it, since many code modules might be involved in the flow in which the bug has occurred, further delaying the time required for testing the software.
- Longer Deployment Times: Since the complete product is released in one go, it might take a longer time to get the system up-and-running in the production environment.
Working of a Continuous Integration Pipeline
The typical workflow of the development to deployment phase in a CI pipeline is as follows: <ref name="Martin">An Article on Continuous Integration by Martin Fowler | 1 May, 2006</ref><ref name="ThoughtWorks">ThoughtWorks-Continuous Integration</ref>
Source - Wikipedia<ref>Continuous Delivery (Wikipedia)</ref>
1. The Developers, who are a part of the Delivery team, will take the current code base from a shared code repository, which is maintained for their project in a version control system.
2. The Developers will then write their own code on top of the current code base taken from the shared code repository.
3. When the changes are done, the developers integrate their code with the shared code repository. This should happen frequently (at least once a day).
4. An automated build is run, every time a developer integrates his changes. Unit tests and Integration tests are also performed in this phase. If either of the build or the tests fail, the development team is notified to fix the errors. The development team is also notified on a successful build.
5. Once the build and the above tests succeed, the software is automatically deployed to a testing environment where more rigorous testing can be done.
6. If the in-depth testing succeeds, the software product with the new functionality becomes release-ready. If the testing fails, the development team is notified to fix the errors. The development team is also notified if the tests are successful.
7. As soon as the customer requires the new functionality in the software, the release-ready build is deployed into the production environment.
Advantages of Continuous Integration Pipelines
CI pipelining significantly reduces the risks posed by taking the traditional approach during the development to deployment phase of a software. By implementing CI pipelining in software projects the following advantages are achieved:<ref name="Martin"></ref><ref name="ThoughtWorks"></ref><ref>Multi-Stage Continuous Integration</ref>
- If any bugs are present in the code, they are detected early. Hence, the effort needed in removing the bug reduces significantly, since a comparatively smaller part of the system is affected and the concerned developers have recently worked on the code, so it is fresh in their minds.
- Integration, testing and deployment of comparatively smaller code is easier, less time-consuming and less risky.
- CI pipelining ensures that a “stable and working” build of the software is available at all times.
- CI pipelining ultimately helps the company to quickly release bugs fixes and enhancements to the customers of their software product.
- CI pipelining makes continuous deployment of the software to the production environment easier. Since, continuous deployment helps customers get bug fixes and enhancements to the software frequently, developers get continuous feedback on the bug fixes and enhancements that they have made to the software. Thus, customers and developers get an opportunity to interact frequently, which is one of the most important factors in developing a good software.
Continuous Integration Tools
Some of the most commonly used continuous integration tools are as follows:
CruiseControl
CruiseControl is a free, open-source, continuous integration tool. CruiseControl is written in Java but is used on a wide variety of projects. Its web-interface provides the users with the details and the current status of each build. Its configuration is written in XML.<ref name="cc">CruiseControl Tool</ref><ref name="compare">The Best Continuous Integration Tools by Vlad Kofman | 18 February, 2009</ref>
Advantages:<ref name="cc"></ref>
- CruiseControl contains many plug-ins for publishing notifications regarding the status of the build via e-mail and instant messaging.
- Build results in CruiseControl, are color-coded so that the users are provided with instant visual feedback about the status of their build.
Disadvantages:<ref name="compare"></ref>
- CC does not have a configuration User Interface. It requires editing the XML files directly to change configurations.
- The User Interface of CC is very primitive.
- The learning curve for CC is high.
Apache Continuum
Apache Continuum is an enterprise-ready continuous integration server. It has a web-based user-interface that helps in monitoring the current status of a build. Continuum comes as a standalone application and also as a WAR file. <ref name="cont">Apache Continuum Tool/</ref><ref>What is Continuum and how do i install it? by Deron Eriksson</ref>
Advantages: <ref name="compare"></ref><ref name="ibm">Choosing a Continuous Integration Server by Paul Duvall | 5 September, 2006</ref>
- Continuum offers Role-based security. This feature gives the administrator control over what aspects of the build can be seen by different users.
- Setup and Configuration of Continuum is fairly straight forward.
- The learning curve for Continuum is low.
Disadvantages: <ref>CI Tool Comparison 2 April, 2010 </ref>
- Continuum does not have a very wide variety of plug-ins and extensions.
Travis CI
Travis is an open-source, continuous integration service which is used for GitHub Projects. Travis supports development in many languages such as C, C++, Java, Ruby etc. Several open source projects including Ruby-on-Rails are using Travis. Travis is configured by adding a .travis.yml file to the root directory of a GitHub repository.<ref>Travis CI (Wikipedia)</ref>
Advantages:<ref>CI with Travis by Kamran Ayub</ref>
- Travis CI provides real time logging. You can watch the output of the builds continuously.
- Travis CI’s user-interface is fairly easy to use.
Disadvantages:<ref>A Blog on Travis-CI by Kai Inkinen | 7 October, 2013</ref>
- Travis CI is free only for open source projects. For all non-public projects, a user needs to use Travis Pro for a subscription fee.
Go
Go is an open source tool, built by ThoughtWorks Studios, that is used to support continuous delivery. It helps create and manage automated deployment pipelines. It supports automating the entire build-test-release process from check-in to deployment.<ref>Open-Sourcing ThoughtWorks Go by Martin Fowler | 25 March, 2014</ref>
Advantages: <ref>Continuous Delivery with Go by Robert Valk | 30 June 2014</ref>
- The installation of Go is very quick and easy.
- The administrative User-Interface of Go is very refined.
Disadvantages: <ref>Discussion on Go</ref>
- The learning curve for Go is higher.
- The custom e-mail notifications in Go are not configurable enough.
Hudson
Hudson is a free continuous integration tool written in Java. It supports various SCM tools including CVS, Subversion, Git, Perforce, Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. Hudson became a popular alternative to CruiseControl and other open-source build servers in 2008.<ref>Hudson (Wikipedia)</ref>
Advantages:<ref name="compare"></ref>
- Hudson has a very short learning curve.
- The User-Interface of Hudson is very refined.
- The entire configuration can be done via its User-Interface.
- It can be adapted to be used with many development languages.
Disadvantages:<ref>Pros/Cons of CI Systems - A StackOverflow Discussion</ref>
- There is no way in Hudson to mass manage the build results.
- The web User-Interface of Hudson sometimes gets slower.
- Some of Hudson’s plug-ins eat a lot of memory.
Challenges in a Continuous Integration Pipeline Implementation
Some of the challenges involved in implementing a CI pipeline are as follows:<ref>CI Practices and Challenges by Cherifa Mansoura | 10 March 2013</ref>
- The infrastructure (hardware for the build machines, a version control system etc.) required for implementing a continuous integration pipeline may be very costly, especially for small software projects.
- Some practices involved in CI, like integrating code frequently with the shared code repository, require a lot of discipline on the part of the developers. It might not always be possible to maintain this discipline especially in larger Projects, which may involve a large number of developers working together on many components of the software.
See Also
References
<references/>
Further Reading
- An article on Continuous Integration By Mike Jackson
- A video on Continuous Integraion by PHP UK Conference | 2013
- An article on Robust continuous integration environment By Shanky Town
- CI- Pipeline case study By IndicThreads
- The Original Article on continuous integration By Martin Fowler