CSC/ECE 517 Fall 2014/ch1a 16 va: Difference between revisions
No edit summary |
|||
Line 1: | Line 1: | ||
'''<big>Continuous Integration Pipelines</big>''' | |||
[[File:Continuous_Delivery_process_diagram.png|right|frame|A Typical | |||
Continuous Integration Pipeline | |||
<ref>http://en.wikipedia.org/wiki/Continuous_delivery</ref>]] | |||
__TOC__ | |||
== Introduction == | == Introduction == | ||
Continuous Integration Pipeline is a software development technique, which | |||
focuses at automating major portions of the build, test and deployment | |||
tasks. In CI pipelining, any change made by the developer to their | |||
projects’ shared source code repository automatically triggers a build. | |||
Later on, when the build and automated unit-tests are successful, the | |||
application is automatically deployed to a production-like environment | |||
where more rigorous testing can be done. | |||
<ref>http://www.asp.net/aspnet/overview/developing-apps-with-windows- | |||
azure/building-real-world-cloud-apps-with-windows-azure/continuous- | |||
integration-and-continuous-delivery</ref>.This technique allows the | |||
contributors of the project to build, test and deploy their code, easily | |||
and quickly, that too with greatly reduced risk. Thus, the CI pipelining | |||
ultimately ensures the rapid delivery of a quality software product.<br> | |||
== Drawbacks of the Traditional Software Development Practices == | |||
Consider the following workflow that demonstrates the development to | |||
deployment phase in a traditional software development life-cycle:<br> | |||
<br> | |||
1) Initially, the developers will code their individual modules.<br> | |||
2) Once the development is completed, the various modules are | |||
integrated together.<br> | |||
3) Once the merge is completed, the entire product is deployed in the | |||
testing environment.<br> | |||
4) Once the testing is completed, the product becomes release ready | |||
and is later on deployed in the production environment.<br> | |||
<br> | |||
Generally, the above workflow is very risky to follow, since many severe | |||
issues may arise in above mentioned steps. These are stated below:<br> | |||
<br> | |||
1) Longer Integration Times: The code written by different developers | |||
might not integrate successfully with each other, resulting in build | |||
failures. Since, many code modules are integrated in one go, it is very | |||
difficult and time consuming to detect and resolve the cause of the build | |||
failure.<br> | |||
2) Longer Testing Times: Once the complete product is ready, it may | |||
take a lot of time to perform the complete end-to-end testing of the | |||
software. Also, if a bug arises in the code, it may be difficult to | |||
resolve it, since many code modules might be involved in the flow in which | |||
the bug has occurred, further delaying the time required for testing the | |||
software.<br> | |||
3) Longer Deployment Times: Since the complete product is released in | |||
one go, it might take a longer time to get the system up-and-running in | |||
the production environment.<br> | |||
== Working of a Continuous Integration Pipeline == | |||
CI pipelining significantly reduces the risks posed by taking the | |||
traditional approach during the development to deployment phase of a | |||
software. It also reduces the time involved in resolving these issues.<br> | |||
<br> | |||
The typical workflow of the development to deployment phase in a CI | |||
pipeline is as follows: <ref | |||
name="Martin">http://www.martinfowler.com/articles/continuousIntegration.h | |||
tml</ref><ref name="ThoughtWorks">http://www.thoughtworks.com/continuous- | |||
integration</ref><br> | |||
<br> | |||
1) Developers will take the current code base from a shared code | |||
repository (which is maintained for their project) on which they will | |||
write their own code. <br> | |||
2) When the changes are done, the developers integrate their code | |||
with the shared code repository. This should happen frequently (at least | |||
once a day).<br> | |||
3) An automated build is run, every time a developer integrates his | |||
changes. Unit tests and Integration tests are also performed in this | |||
phase. If either of the build or the tests fail, the development team is | |||
notified to fix the errors. The development team is also notified on a | |||
successful build.<br> | |||
4) Once the build and the above tests succeed, the software is | |||
automatically deployed to a testing environment where more rigorous | |||
testing can be done.<br> | |||
5) If the in-depth testing succeeds, the software product with the | |||
new functionality becomes release-ready. If the testing fails, the | |||
development team is notified to fix the errors. The development team is | |||
also notified if the tests are successful.<br> | |||
6) As soon as the customer requires the new functionality in the | |||
software, the release-ready build is deployed into the production | |||
environment.<br> | |||
== Advantages of Continuous Integration Pipelines == | |||
By implementing CI pipelining in software project the following advantages | |||
are achieved:<ref name="Martin"></ref><ref | |||
name="ThoughtWorks"></ref><ref>http://en.wikipedia.org/wiki/Multi- | |||
stage_continuous_integration</ref><br> | |||
<br> | |||
1) If any bugs are present in the code, they are detected early. | |||
Hence, the effort needed in removing the bug reduces significantly, since | |||
a comparatively smaller part of the system is affected and the concerned | |||
developers have recently worked on the code, so it is fresh in their | |||
minds. <br> | |||
2) Integration, testing and deployment of comparatively smaller code | |||
is easier, less time-consuming and less risky.<br> | |||
3) CI pipelining ensures that a “stable and working” build of the | |||
software is available at all times.<br> | |||
4) CI pipelining ultimately helps the company to quickly release bugs | |||
fixes and enhancements to the customers of their software product.<br> | |||
5) CI pipelining makes continuous deployment of the software to the | |||
production environment easier. Since, continuous deployment helps | |||
customers get bug fixes and enhancements to the software frequently, | |||
developers get continuous feedback on the bug fixes and enhancements that | |||
they have made to the software. Thus, customers and developers get an | |||
opportunity to interact frequently, which is one of the most important | |||
factors in developing a good software.<br> | |||
== Continuous Integration Tools == | |||
Some of the most commonly used continuous integration tools are as | |||
follows: <br> | |||
=== CruiseControl === | |||
[[File:CruiseControl.jpg|right|frame|Build Status Page in CruiseControl | |||
<ref name="cc"></ref>]] | |||
CruiseControl is a free, open-source, continuous integration tool. | |||
CruiseControl is written in Java but is used on a wide variety of | |||
projects. Its web-interface provides the users with the details and the | |||
current status of each build. Its configuration is written in XML.<ref | |||
name="cc">http://cruisecontrol.sourceforge.net/index.html</ref><ref | |||
name="compare">http://www.developer.com/open/article.php/3803646/The- | |||
Best-Continuous-Integration-Tools.htm</ref><br> | |||
<br> | |||
Advantages:<ref name="cc"></ref><br> | |||
<br> | |||
1) CruiseControl contains many plugins for publishing notifications | |||
regarding the status of the build via e-mail and instant messaging. <br> | |||
2) Build results in CruiseControl, are color-coded so that the users | |||
are provided with instant visual feedback about the status of their build. | |||
<br> | |||
<br> | |||
Disadvantages:<ref name="compare"></ref><br> | |||
<br> | |||
1) CC does not have a configuration User Interface. It requires | |||
editing the XML files directly to change configurations. <br> | |||
2) The User Interface of CC is very primitive. <br> | |||
3) The learning curve for CC is high. <br> | |||
=== Apache Continuum === | |||
[[File:ApacheContinium.png|right|frame|Apache Continuum's Build Status | |||
Screen | |||
<ref>http://continuum.apache.org/docs/1.4.2/user_guides/report/index.html< | |||
/ref>]] | |||
Apache Continuum is an enterprise-ready continuous integration server. It | |||
has a web-based user-interface that helps in monitoring the current status | |||
of a build. Continuum comes as a standalone application and also as a war | |||
file. <ref | |||
name="cont">http://continuum.apache.org/</ref><ref>http://www.avajava.com/ | |||
tutorials/lessons/what-is-continuum-and-how-do-i-install-it.html</ref><br> | |||
<br> | |||
Advantages: <ref name="compare"></ref><ref | |||
name="ibm">http://www.ibm.com/developerworks/library/j-ap09056/</ref><br> | |||
<br> | |||
1) Continuum offers Role-based security. This feature gives the | |||
administrator control over what aspects of the build can be seen by | |||
different users. <br> | |||
2) Setup and Configuration of Continuum is fairly straight forward. <br> | |||
3) The learning curve for Continuum is low. <br> | |||
<br> | |||
Disadvantages:<ref>http://www.thinkplexx.com/learn/article/build- | |||
chain/ci/choosing-continuous-integrationci-tool-opensource-vs- | |||
professional-cruisecontrol-hudson-continuum-vs-teamcity</ref><br> | |||
<br> | |||
1) Continuum does not have a very wide variety of plugins and | |||
extensions.<br> | |||
=== Travis CI === | |||
[[File:HudsonFailedBranch.png|right|frame|Travis CI's Real Time Logging | |||
Screen <ref>http://railsware.com/blog/2013/09/10/capistrano-recipe-for- | |||
checking-travis-ci-build-status/</ref>]] | |||
Travis is an open-source, continuous integration service which is used for | |||
GitHub Projects. Travis supports development in many languages such as C, | |||
C++, Java, Ruby etc. Several open source projects including Ruby-on-Rails | |||
are using Travis. Travis is configured by adding a .travis.yml file to the | |||
root directory of a GitHub | |||
repository.<ref>http://en.wikipedia.org/wiki/Travis_CI</ref><br> | |||
<br> | |||
Advantages:<ref>http://tech.pro/tutorial/1749/get-your-ci-on-with-travis- | |||
ci</ref><br> | |||
<br> | |||
1) Travis CI provides real time logging. You can watch the output of the | |||
builds continuously.<br> | |||
2) Travis CI’s user-interface is fairly easy to use.<br> | |||
<br> | |||
Disadvantages:<ref>https://blog.futurice.com/tech-pick-of-the-week- | |||
travis-ci</ref><br> | |||
<br> | |||
1) Travis CI is free only for open source projects. For all non-public | |||
projects, a user needs to use Travis Pro for a subscription fee.<br> | |||
=== Go === | |||
[[File:Go.png|right|frame|A Go Pipeline | |||
<ref>https://jazz.net/library/content/articles/rtc/2.0/build- | |||
hudson/images/8-27-2009%2010-54-23%20AM.png</ref>]] | |||
Go is an open source tool, built by ThoughtWorks Studios, that is used to | |||
support continuous delivery. It helps create and manage automated | |||
deployment pipelines. It supports automating the entire build-test-release | |||
process from check-in to | |||
deployment.<ref>http://martinfowler.com/articles/go- | |||
interview.html</ref><br> | |||
<br> | |||
Advantages: <ref>http://www.sixtree.com.au/articles/2014/continuous- | |||
delivery-with-go/</ref><br> | |||
<br> | |||
1) The installation of Go is very quick and easy. <br> | |||
2) The administrative User-Interface of Go is very refined. <br> | |||
<br> | |||
Disadvantages: <ref>https://groups.google.com/forum/#!searchin/go- | |||
cd/jenkins/go-cd/SlfYlFjnA7o/s4tAuDkNW3sJ</ref><br> | |||
<br> | |||
1) The learning curve for Go is higher.<br> | |||
2) The custom e-mail notifications in Go are not configurable enough.<br> | |||
=== Hudson === | |||
[[File:Hudson.png|right|frame|Hudson's Build Status Screen | |||
<ref>http://marcovaltas.com/img/posts/blog_pipeline.png</ref>]] | |||
Hudson is a free continuous integration tool written in Java. It supports | |||
various SCM tools including CVS, Subversion, Git, Perforce, Clearcase and | |||
RTC, and can execute Apache Ant and Apache Maven based projects, as well | |||
as arbitrary shell scripts and Windows batch commands. Hudson became a | |||
popular alternative to CruiseControl and other open-source build servers | |||
in 2008.<ref>http://en.wikipedia.org/wiki/Hudson_(software)</ref><br> | |||
<br> | |||
Advantages:<ref name="compare"></ref><br> | |||
<br> | |||
1) Hudson has a very short learning curve.<br> | |||
2) The User-Interface of Hudson is very refined.<br> | |||
3) The entire configuration can be done via its User-Interface.<br> | |||
4) It can be adapted to be used with many development languages.<br> | |||
<br> | |||
Disadvantages:<ref>http://stackoverflow.com/questions/386697/pro-con- | |||
continuous-integration-systems</ref><br> | |||
<br> | |||
1) There is no way in Hudson to mass manage the build results.<br> | |||
2) The web User-Interface of Hudson sometimes gets slower.<br> | |||
3) Some of Hudson’s plugins eat a lot of memory.<br> | |||
== | == References == | ||
<references/> |
Revision as of 22:40, 19 September 2014
Continuous Integration Pipelines
Introduction
Continuous Integration Pipeline is a software development technique, which
focuses at automating major portions of the build, test and deployment
tasks. In CI pipelining, any change made by the developer to their
projects’ shared source code repository automatically triggers a build.
Later on, when the build and automated unit-tests are successful, the
application is automatically deployed to a production-like environment
where more rigorous testing can be done.
<ref>http://www.asp.net/aspnet/overview/developing-apps-with-windows-
azure/building-real-world-cloud-apps-with-windows-azure/continuous-
integration-and-continuous-delivery</ref>.This technique allows the
contributors of the project to build, test and deploy their code, easily
and quickly, that too with greatly reduced risk. Thus, the CI pipelining
ultimately ensures the rapid delivery of a quality software product.
Drawbacks of the Traditional Software Development Practices
Consider the following workflow that demonstrates the development to
deployment phase in a traditional software development life-cycle:
1) Initially, the developers will code their individual modules.
2) Once the development is completed, the various modules are
integrated together.
3) Once the merge is completed, the entire product is deployed in the
testing environment.
4) Once the testing is completed, the product becomes release ready
and is later on deployed in the production environment.
Generally, the above workflow is very risky to follow, since many severe
issues may arise in above mentioned steps. These are stated below:
1) Longer Integration Times: The code written by different developers
might not integrate successfully with each other, resulting in build
failures. Since, many code modules are integrated in one go, it is very
difficult and time consuming to detect and resolve the cause of the build
failure.
2) Longer Testing Times: Once the complete product is ready, it may
take a lot of time to perform the complete end-to-end testing of the
software. Also, if a bug arises in the code, it may be difficult to
resolve it, since many code modules might be involved in the flow in which
the bug has occurred, further delaying the time required for testing the
software.
3) Longer Deployment Times: Since the complete product is released in
one go, it might take a longer time to get the system up-and-running in
the production environment.
Working of a Continuous Integration Pipeline
CI pipelining significantly reduces the risks posed by taking the
traditional approach during the development to deployment phase of a
software. It also reduces the time involved in resolving these issues.
The typical workflow of the development to deployment phase in a CI
pipeline is as follows: <ref
name="Martin">http://www.martinfowler.com/articles/continuousIntegration.h
tml</ref><ref name="ThoughtWorks">http://www.thoughtworks.com/continuous-
integration</ref>
1) Developers will take the current code base from a shared code
repository (which is maintained for their project) on which they will
write their own code.
2) When the changes are done, the developers integrate their code
with the shared code repository. This should happen frequently (at least
once a day).
3) An automated build is run, every time a developer integrates his
changes. Unit tests and Integration tests are also performed in this
phase. If either of the build or the tests fail, the development team is
notified to fix the errors. The development team is also notified on a
successful build.
4) Once the build and the above tests succeed, the software is
automatically deployed to a testing environment where more rigorous
testing can be done.
5) If the in-depth testing succeeds, the software product with the
new functionality becomes release-ready. If the testing fails, the
development team is notified to fix the errors. The development team is
also notified if the tests are successful.
6) As soon as the customer requires the new functionality in the
software, the release-ready build is deployed into the production
environment.
Advantages of Continuous Integration Pipelines
By implementing CI pipelining in software project the following advantages
are achieved:<ref name="Martin"></ref><ref
name="ThoughtWorks"></ref><ref>http://en.wikipedia.org/wiki/Multi-
stage_continuous_integration</ref>
1) If any bugs are present in the code, they are detected early.
Hence, the effort needed in removing the bug reduces significantly, since
a comparatively smaller part of the system is affected and the concerned
developers have recently worked on the code, so it is fresh in their
minds.
2) Integration, testing and deployment of comparatively smaller code
is easier, less time-consuming and less risky.
3) CI pipelining ensures that a “stable and working” build of the
software is available at all times.
4) CI pipelining ultimately helps the company to quickly release bugs
fixes and enhancements to the customers of their software product.
5) CI pipelining makes continuous deployment of the software to the
production environment easier. Since, continuous deployment helps
customers get bug fixes and enhancements to the software frequently,
developers get continuous feedback on the bug fixes and enhancements that
they have made to the software. Thus, customers and developers get an
opportunity to interact frequently, which is one of the most important
factors in developing a good software.
Continuous Integration Tools
Some of the most commonly used continuous integration tools are as
follows:
CruiseControl
CruiseControl is a free, open-source, continuous integration tool.
CruiseControl is written in Java but is used on a wide variety of
projects. Its web-interface provides the users with the details and the
current status of each build. Its configuration is written in XML.<ref
name="cc">http://cruisecontrol.sourceforge.net/index.html</ref><ref
name="compare">http://www.developer.com/open/article.php/3803646/The-
Best-Continuous-Integration-Tools.htm</ref>
Advantages:<ref name="cc"></ref>
1) CruiseControl contains many plugins for publishing notifications
regarding the status of the build via e-mail and instant messaging.
2) Build results in CruiseControl, are color-coded so that the users
are provided with instant visual feedback about the status of their build.
Disadvantages:<ref name="compare"></ref>
1) CC does not have a configuration User Interface. It requires
editing the XML files directly to change configurations.
2) The User Interface of CC is very primitive.
3) The learning curve for CC is high.
Apache Continuum
Apache Continuum is an enterprise-ready continuous integration server. It
has a web-based user-interface that helps in monitoring the current status
of a build. Continuum comes as a standalone application and also as a war
file. <ref
name="cont">http://continuum.apache.org/</ref><ref>http://www.avajava.com/
tutorials/lessons/what-is-continuum-and-how-do-i-install-it.html</ref>
Advantages: <ref name="compare"></ref><ref
name="ibm">http://www.ibm.com/developerworks/library/j-ap09056/</ref>
1) Continuum offers Role-based security. This feature gives the
administrator control over what aspects of the build can be seen by
different users.
2) Setup and Configuration of Continuum is fairly straight forward.
3) The learning curve for Continuum is low.
Disadvantages:<ref>http://www.thinkplexx.com/learn/article/build-
chain/ci/choosing-continuous-integrationci-tool-opensource-vs-
professional-cruisecontrol-hudson-continuum-vs-teamcity</ref>
1) Continuum does not have a very wide variety of plugins and
extensions.
Travis CI
Travis is an open-source, continuous integration service which is used for
GitHub Projects. Travis supports development in many languages such as C,
C++, Java, Ruby etc. Several open source projects including Ruby-on-Rails
are using Travis. Travis is configured by adding a .travis.yml file to the
root directory of a GitHub
repository.<ref>http://en.wikipedia.org/wiki/Travis_CI</ref>
Advantages:<ref>http://tech.pro/tutorial/1749/get-your-ci-on-with-travis-
ci</ref>
1) Travis CI provides real time logging. You can watch the output of the
builds continuously.
2) Travis CI’s user-interface is fairly easy to use.
Disadvantages:<ref>https://blog.futurice.com/tech-pick-of-the-week-
travis-ci</ref>
1) Travis CI is free only for open source projects. For all non-public
projects, a user needs to use Travis Pro for a subscription fee.
Go
Go is an open source tool, built by ThoughtWorks Studios, that is used to
support continuous delivery. It helps create and manage automated
deployment pipelines. It supports automating the entire build-test-release
process from check-in to
deployment.<ref>http://martinfowler.com/articles/go-
interview.html</ref>
Advantages: <ref>http://www.sixtree.com.au/articles/2014/continuous-
delivery-with-go/</ref>
1) The installation of Go is very quick and easy.
2) The administrative User-Interface of Go is very refined.
Disadvantages: <ref>https://groups.google.com/forum/#!searchin/go-
cd/jenkins/go-cd/SlfYlFjnA7o/s4tAuDkNW3sJ</ref>
1) The learning curve for Go is higher.
2) The custom e-mail notifications in Go are not configurable enough.
Hudson
Hudson is a free continuous integration tool written in Java. It supports
various SCM tools including CVS, Subversion, Git, Perforce, Clearcase and
RTC, and can execute Apache Ant and Apache Maven based projects, as well
as arbitrary shell scripts and Windows batch commands. Hudson became a
popular alternative to CruiseControl and other open-source build servers
in 2008.<ref>http://en.wikipedia.org/wiki/Hudson_(software)</ref>
Advantages:<ref name="compare"></ref>
1) Hudson has a very short learning curve.
2) The User-Interface of Hudson is very refined.
3) The entire configuration can be done via its User-Interface.
4) It can be adapted to be used with many development languages.
Disadvantages:<ref>http://stackoverflow.com/questions/386697/pro-con-
continuous-integration-systems</ref>
1) There is no way in Hudson to mass manage the build results.
2) The web User-Interface of Hudson sometimes gets slower.
3) Some of Hudson’s plugins eat a lot of memory.
References
<references/>