Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2012/ch2a 2w29 st

2012-10-27T04:30:53Z

Sbagga:

<h2>SaaS - 5.1 - Testing overview</h2>
__TOC__
==Introduction==
This ia a textbook section that covers the online lecture on [https://www.youtube.com/watch?v=Hpg9303P0Ts Testing Overview].
Gone are the days when the developers used to write code and toss it to the QA team for testing. Today’s developers are far more responsible for testing their own code. Testing these days is far more automated in the sense that the tester doesn’t need to manually check the output if its correct or not. In this document, we give the history of software testing and introduce some of the newer methods that are being used today such as Behavior Driven Development and Test Driven Development.
==History==
Brian Kernighan famously quoted that “Debugging is twice as hard as writing the code in the first place.” Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
In other words it meant if you’re as clever as you can be when you write your code, how will you ever debug it.

Another famous computer scientist Dijkstra famously remarked that "testing can be used to show the presence of bugs but never to show their absence”.The fact is that NOTHING, not inspection, not formal proof, not testing, can give 100% certainty of no errors. Yet all these techniques, at some cost, can in fact reduce the errors to whatever level you wish.

In the initial stages i.e when the software industry was in a nascent stage, there was no stress upon software testing. In most of the cases there was just a quick check on the software and then the software was handed over to the client.A thorough testing of application was considered to be huge waste of time and resources.This was basically done so that items in the checklist could just be ticked off and software could be developed and handed over to client as soon as possible.However as time passed the importance of software testing was realized. Now software testing is given as much importance as that of developing a software.The change in trend can be easily explained.

A Study conducted in 2002 by NIST reported that software bugs cost the U.S. economy a loss of atleast 59.5 billion dollars annually.It further reports that more than one third of this cost could have been avoided if better software testing was performed. <ref>http://www.abeacha.com/NIST_press_release_bugs_cost.htm</ref>

It is commonly believed that the earlier a defect is found the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found.For example, if a problem in the requirements is found only post-release, then it would cost 10–100 times more to fix than if it had already been found by the requirements review. With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time.

{|class="wikitable"
|-
!Cost to fix a defect
!
!Time detected
!
!
!
!
|-
!
!
!Requirements
!Architecture
!Construction
!System test
!Post-release
|-
!Time introduced
!Requirements
!1×
!3x
!5-10x
!10x
!10-100x
|-
!
!Architecture
!-
!1x
!10x
!15x
!25-100x
|-
!
!Construction
!-
!-
!1x
!10x
!10-25x
|}

Another survey by Electric Cloud, the leading provider of software production management (SPM) solutions, conducted in partnership with Osterman Research showed that the majority of software bugs are attributed to poor testing procedures or infrastructure limitations rather than design problems. Additionally, the software test process is generally considered an unpleasant process, with software development professionals rating the use of their companies’ test systems more painful than preparing taxes.
Fifty-eight percent of respondents pointed to problems in the testing process or infrastructure as the cause of their last major bug found in delivered or deployed software, not design defects.

Specifically, the survey highlighted the following:
Completely automated software testing environments are still rare, with just 12 percent of software development organizations using fully automated test systems. Almost 10 percent reported that all testing was done manually.
Forty-six percent of software developers said they do not have time to test as much as they should.
More than a third of developers, 36 percent, said they do not believe their companies perform enough pre-release testing.
Fifty-three percent said their testing is limited by compute resources.<ref>http://www.electric-cloud.com/news/2010-0602.php</ref>
==Importance of Software Testing==
Importance of Software Testing:

The importance of software testing can be explained in brief in the following points.

'''Error Free Software''': Software testing needs to be done thoroughly to deliver an error free software to the client. Even a very minute mistake in the the software can have a disastrous effect for the client. For example a small failure in a banking software can result in a wrong balance and millions of dollars worth loss to the client or customers of the client. So it is of prime importance that the software delivered to the client be bug free and accurate.

'''Variance from the Requirement''': One important factor while building a software is to adhere to the client requirements. if a software is not built in accordance to the requirements then the software becomes useless and redundant for the client.This is a lot of trouble and overhead for the software developing firm also as the requirement was not well understood.The budget fixed for the development of th software can easily go overboard.In such scenarios verification and validation process come into picture. The two most important questions that needs to be answered are “Is the product being built right” and “Is the right product being built”.If the answer is negative to any one of these questions it means that the the product developed has a variance from the client requirements and necessary changes needs to be made before going ahead with further development.

'''Identify Defect and Prevent their Migration''': If a defect is detected in the requirement analysis stage ,then rectifying the defect is a lot easier and cheaper. But if the defect is not identified and carried over to the next phases of software development then it become more difficult to fix the defects,So it is highly recommended that software testing process be started right when the software development starts.

'''Identify Undiscovered Error''': If proper importance is not given to software testing i.e it is not done thoroughly and just a superficial testing is done there is high probability that some of the errors will creep through to the next phase. In such case using different software testing methodologies help in identifying the hidden errors.Exploratory testing is one such method. In such case the tester randomly tests the software for bugs and finds out the error.
Use the Software in Real Time Environment: Testing a software in development and running it in production are two completely different scenarios.When a developer develops a software he tests it only in a development environment. There is high probability that the software will fail miserably in real time environment.

'''Do Away with Developer Bias''': When a person is designated a tester role his prime responsibility is to test the software. The element of bias is removed. For example when we have a developer testing a software in majority of the cases he is going to be biased towards the software. This is because he has developed the software and its natural human instinct to think one’s product is the best. In such scenario many problems will remain unearthed.

'''Provide Confidence in the Software''': Software testing is also used for asserting the confidence on a developed product. There is a huge difference between being an usable product to operable product. When a software is tested time and over again with a huge degree of success then one can easily approve of the quality of the software developed.
<ref>http://www.cbwc-ontario.org/importance-of-software-testing.html</ref>

==Testing Today==
There are various software development life cycle methodologies available for executing software development projects.Each methodology is unique i.e it is designed for a special purpose and compared to other methodologies has its own advantages and disadvantages.But most methodologies work by dividing the entire life cycle into phases and then sharing tasks across this phases.The common methodologies used for software development and their relationship with respect to testing can be summarized below:

<h4>The Waterfall model:</h4>
It is one of the most common and earliest structured models for software development.Waterfall models stresses that one should only move to a phase after the completing and perfecting the preceding phase.As a result of this the phases of software development in waterfall model are unique and there is no jumping back and forth between the various stages.

A typical waterfall model consists of the following sequential phases

It consists of the following sequential phases through which the development life cycle progresses:
<ref>http://www.guru99.com/testing-methodology.html</ref>

[[File: Waterfall_model.png]]

* Requirement analysis: In this phase, software requirements are captured in such a fashion that they can be translated into actual use cases for the system. The requirements can be derived from performance goals, use cases, target deployment, etc.
* System design: In this phase,the interacting components that make up the system are identified, The exposed interfaces and the communication between them,key algorithms and sequence of interaction are defined, In the end of this phase an architecture and design review is conducted to ensure that the design conforms to the previously defined requirements.
* Implementation : This phase is also termed as Coding and unit testing. In this phase,coding is done for the modules that build the system.Code can also be reviewed and functionality of each module individually tested.
* Verification: Also termed as Integration and system testing. In this phase,all the modules in the system are integrated together and tested as a single system for all the use cases.The primary emphasis is upon making sure that all the modules meet the requirements.
* Deployment and maintenance. This is the final phase where the software system is deployed into the production environment. Any errors identified in this phase are corrected and functionality is added/modified to the system based on the updated requirements.

Waterfall model has the following advantages:
* The life cycle can be compartmentalized into various phases which helps in planning the resources and the amount of effort required through the development process.
* Testing is enforced in every stage in the form of unit testing and reviews.During various stages of the lifecycle different form of reviews like design and code reviews and various forms of testing like unit and integration testing are performed.
* After each phase of lifecycle expectations for deliverables can be set.

Waterfall model has the following disadvantages:
* There is no working version of software until its late in the life cycle.So problems can’t be detcted until the system testing phase.Problems are always hard to fix in the latter phases of life cycle.
* Also for a phase to get started the previous phase must be complete.A system design principle cannot start until the requirement analysis phase is complete and the requirements are frozen. So waterfall model cannot accommodate uncertainties that that may persist after a phase is over.This can lead to delays and extended project schedules.

Testing approach:
During the requirements phase the project requirements are completely defined.Simultaneously the test team brainstorms the scope of testing,test strategy and drafts a detailed test plan.

<h4>Incremental or Iterative Development</h4>

The fundamental principle behind incremental or iterative development is to break down the project into small parts. When one iteration is complete a new module is completed or an existing module is improved. The module is then integrated into the structure and finally the structure is then tested as a whole. In the iterative development model a project is usually broken down into 12 iterations of one to four week duration each.Finally the system is tested at the end of each duration and the test feedback is immediately incorporated. Time spent on the successive iterations can be modified on the basis of experience gained from past iterations. So the system grows by incorporating new functions during the development portion of each iteration.Each iteration thus involves adding of new functions during the development phase.testing evolves as the system evolves.
<br>
<br>
[[File:Iterative_dev.jpg]]
<br>
<br>
The main advantage of iterative development model is that corrective actions can be taken at end of each iteration.

The main disadvantages of iterative development model are:
* Each iteraton involved giving feedback about the deliverables,timelines,efforts and so on.SO the overhead is considerably higher.
* It is hard to freeze the requirements as requirements may need change based on feedback and increasing customer demands.This can lead to more number of iterations and thus delay in deliverables.
* An efficient control change mechanism is needed to manage the system changes made during each iteration.

Testing approach:
As soon as iteration is complete the entire system is subjected to testing.The feedback from testing is immediately available and further incorporated into the next cycle.Testing time required for the successive iterations can be reduced based on the experience gained from past iterations.

<h4>Agile Methodology</h4>
Previously Majority of the software development life cycle methodologies could be categorised into either iterative or sequential model like waterfall model does.But as software systems evolved and became more complex both of these models couldn’t efficiently adapt to the significant and continuous number of changes.Agile methodology was developed to solve this issue.It was develoepd to respond to changes quicly and smoothly. The drawback with iterative model was that even though it removed the disadvantage of sequential models it was still based on the waterfall model.In Agile methodology , software is developed in incremental, rapid cycles. Interactions amongst customers, developers and client are emphasized rather than processes and tools. Agile methodology focuses on responding to change rather than extensive planning.
<br>
<br>
[[File: Agile_dev.gif]]
<br>
<br>
The main differences between agile and traditional methodologies are as follows:
* An incremental method of development is followed rather than the traditional sequential method.There are small incremental releases and each release is based on previous functionality.
* Each release is thoroughly tested and that helps to ensure that the defects are addressed in the next iteration.
* There is more emphasis given on people and their interactions rather than processes and tools.The developers,customers and testers continuously interact with each other.This interaction ensures that the tester us aware of the features being developed during a particular iteration and so can easily identify any sort of discrepancy between the system and the requirements.
* More priority is given to working software rather than detailed documentation. Agile methodologies rely on face-to-face communication and collaboration, with people working in pairs. As there is extensive communication between customers and team members, there is no need for comprehensive requirements document. All agile projects have customers as an integral part of the team.When developers have a query regarding program requirements thy can get it immediately clarified from the customers.

The disadvantage is that a constant client interaction leads to and added time pressure on all stakeholders including the client themselves , software development and test teams .

Testing approach:
Incremental testing approach is followed and thus every release of the project is tested thoroughly.This ensures that any bugs in the system are fixed before the next release.

<h4>Extreme Programming</h4>
Extreme programming<ref>http://xprogramming.com/book/whatisxp</ref> is a form of agile methodology that believes in short development cycles.So rather than designing the whole of the system at the start of the project the preliminary design work is shortened down to solve the simple tasks that have already been identified.The developers have to interact frequently with customers and other developers. A simple task is started and as soon as it is developed customer feedback is taken.The system is delivered to the customer as soon as possible and the requirements are then refined on the basis of customer feedback.So the requirements evolve over a period of time and developers are able to respond to changes quickly.Extreme programming emphasizes on pair programming.This means one developer writes the code for a particular feature and the other developer reviews it. In theory, the driver focuses on the code at hand: the syntax, semantics, and algorithm. The navigator focuses less on that, and more on a level of abstraction higher: the test they are trying to get to pass, the technical task to be delivered next, the time elapsed since all the tests were run, the time elapsed since the last repository commit, and the quality of the overall design. The theory is that pairing results in better designs, fewer bugs, and much better spread of knowledge across a development team, and therefore more functionality per unit time, measured over the long term.
[[File:Extreme_prog.jpg]]

<br>
Extreme Programming is highly useful in the many situations.

* If a customer doesn't have a clear understanding of the system then the developers can interact continuously with the customer ,deliver small pieces and ask the customer for feedback.Corrective action is then taken.
* If a technology used to develop a system is significantly new and its a completely new platform then frequent test cycles in extreme programing mitigate the risk of incompatibility with other existing systems.
* If you want automated unit and functional tests there may be a need to change system design such that each module can be tested in isolation sing automation. XP(Extreme programming) comes handy in such scenario.

The main advantage of following XP is that customers having a vague software design in mind can go ahead to implement their product. The continuous testing and integration ensures that the software code delivered is of the highest standards.

Testing approach:
Extreme programming follows a test driven development.It is explained in brief in the subsequent section.

==Test Driven Development (TDD)==
Test-driven development is one of the core practices of Extreme Programming. Test cases are written first and then code is written to pass the existing test cases. Then new test cases are added to test the existing functionality, and then the entire test suite is run to ensure that the code fails. Then new functionality is added or existing functionality is modified so that the code can withstand the failed test cases. This cycle continues until the test code passes all of the test cases that the team can create. The code is then refactored to make it DRY and more maintainable.

Test-driven development is totally non conventional in the sense that instead of writing code first and then testing it, you write the tests first and then write code to make the tests pass. This is done iteratively. Only when one test case passes, the developer moves on to the next test case and writes code to make it pass. This process is continued until all tests pass.

With test-driven development we can even start with an unclear set of requirements and then can discuss with the customer later about new requirements or change in existing requirements. Even if the code is not ready for a particular functionality but the tests are written, it will ensure that the functionality is addressing all the requirements given by the customer and unnecessary functionality is not included. It allows you to build your software step-by-step and then as the requirements become more clear it evolves the system.<br>
Other advantages of TDD:
* Loosely coupled and highly cohesive code is promoted by Test-driven development because the functionality is evolved in small steps.
* The tests that we write can act as documentation for the final system’s specifications.
* Time for retesting is reduced because of automated testing. So we don’t have to waste time in retesting existing functionality.
* You know exactly what you have to do in order to make a test pass. Your effort can be measured in terms of number of tests passed.
Unit testing is also ensured through Test-driven development. We will still need to all other kinds of testing such as acceptance testing, system integration testing etc.

<h4>Steps in Test-Driven Development</h4>
The following figure shows the steps involved in test-driven development process
<ref>http://msdn.microsoft.com/en-us/library/ff649520.aspx</ref>
<br>
[[File:TDD.gif]]
<br>
==Behavior Driven Development (BDD)==
Behavior Driven Development provides a common language between developers, analysts, and customers, thereby, reducing miscommunication between which was common in earlier forms of software development. This language is used by developers to communicate while developing and explaining code. It makes sure that the customers and the developers have a common vocabulary while talking about the system being developed.

While TDD makes sure that the technical quality of software is up to the mark, behavior-driven development makes sure that the needs of the customer are fulfilled. TDD takes care of the verification part i.e. building the thing right, BDD takes care of the validation part i.e. building the right thing.

Building the Right Thing :
BDD ensures that the systems is built according to the requirements of the customer and is delivered on time. By having a common language between the stakeholders and the developers we have less chance of going wrong with implementing the requirements. This common language is defining the requirement in terms of user stories.

Reducing Risk :
If we are adhering to Behavior Driven Development and our focus is to make a test pass than it is highly unlikely that a developer will go off the way and develop something unnecessary.And with a suite of acceptance tests developed in partnership with the stakeholder, the customer can monitor the system throughout its lifecycle.

With BDD we create a common language which is shared by all stakeholders. It is perfect for projects where the requirements are changing rapidly or are not specified well. Also, it is easier to estimate the effort needed to complete the project because it can be divided into small chunks of user stories.

One of the disadvantages with BDD is that it can be very expensive to maintain constant contact with the customer. Another disadvantage is that we may be able to satisfy all the customer requirements but end up with bad software architecture which would hurt later while maintaining the code.
<ref>Aramando, Fox (2012). Engineering Long Lasting Software Pg 153-155. San Francisco: Strawberry Canyon</ref>
<ref>http://my.safaribooksonline.com/book/software-engineering-and-development/software-testing/9781449309718</ref>
<ref>http://bubusdaybook.blogspot.com/2011/08/extreme-programming-in-nutshell.html</ref>

==Cucumber and RSpec==
Using cucumber, we can describe, in plain text, how a software should behave. It executes plain-text functional descriptions as automated tests. Cucumber supports Behavior Driven Development. The tests are written before the code is written and is verified by non technical stakeholders. The production code is then written to make the stories pass.
<ref> https://github.com/cucumber/cucumber/wiki</ref>

Following is an example of a cucumber scenario in the BackChannel app:
<pre>
Feature: User can add post

Scenario: Add a post
Given I am on the posts index page
When I follow “Add new post”
Then I should be on the Create New Post page
When I fill in “Title” with “Fight Club”
And I fill in “Content” with “The things you own, end up owning you”
And I press “Save Changes”
Then I should be on the posts index page
And I should see “Fight Club”
</pre>

There are six steps to behavior driven development using Cucumber and Rspec<ref>http://cukes.info/</ref>
# Describe behavior in plain text using Cucumber
# Write a step definition in ruby using Rspec
# Run the test and it will fail because the code is not been written yet
# Write code to make the step pass
# Run the test again and see the step pass
# Repeat 2-5 until all steps pass in the behavior
<br>
[[File:Cucumber.jpg|Cucumber and RSpec loop]]
<br>
The test driven development loop is also known as Red-Green-Refactor.
<ref>Aramando, Fox (2012). Engineering Long Lasting Software Pg 164-165. San Francisco: Strawberry Canyon</ref>
<br>
Red step: Run the test and verify that it fails because you haven’t yet implemented the code.<br>
Green step: Write the simplest possible code that causes this test to pass without breaking any existing tests.<br>
Refactor step: Refactor the code if there is any scope of refactoring.<br>

==Conclusion==
BDD and TDD may seem strange at first but once you use it you realize you have been using these techniques in conventional development also, while doing unit testing. Rather than coding first and then debugging the code to find the problem, TDD is much better of way of developing a system in that you can isolate the problem really easily because you whole is divided into features and specs.

If we write code first and then debug, we end up using the same techniques as TDD but less efficiently and less productively. Using TDD, bugs can be spotted quickly and regression testing is easy because all the testing is automated.

==References==
<references/>

The video of the lecture can be found [https://www.youtube.com/watch?v=Hpg9303P0Ts here].

CSC/ECE 517 Fall 2012/ch2a 2w29 st

2012-10-27T00:34:58Z

Sbagga:

<h2>SaaS - 5.1 - Testing overview</h2>
__TOC__
==Introduction==
This ia a textbook section that covers the online lecture on [https://www.youtube.com/watch?v=Hpg9303P0Ts Testing Overview].
Gone are the days when the developers used to write code and toss it to the QA team for testing. Today’s developers are far more responsible for testing their own code. Testing these days is far more automated in the sense that the tester doesn’t need to manually check the output if its correct or not. In this document, we give the history of software testing and introduce some of the newer methods that are being used today such as Behavior Driven Development and Test Driven Development.
==History==
Brian Kernighan famously quoted that “Debugging is twice as hard as writing the code in the first place.” Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
In other words it meant if you’re as clever as you can be when you write your code, how will you ever debug it.

Another famous computer scientist Dijkstra famously remarked that "testing can be used to show the presence of bugs but never to show their absence”.The fact is that NOTHING, not inspection, not formal proof, not testing, can give 100% certainty of no errors. Yet all these techniques, at some cost, can in fact reduce the errors to whatever level you wish.

In the initial stages i.e when the software industry was in a nascent stage, there was no stress upon software testing. In most of the cases there was just a quick check on the software and then the software was handed over to the client.A thorough testing of application was considered to be huge waste of time and resources.This was basically done so that items in the checklist could just be ticked off and software could be developed and handed over to client as soon as possible.However as time passed the importance of software testing was realized. Now software testing is given as much importance as that of developing a software.The change in trend can be easily explained.

A Study conducted in 2002 by NIST reported that software bugs cost the U.S. economy a loss of atleast 59.5 billion dollars annually.It further reports that more than one third of this cost could have been avoided if better software testing was performed. <ref>http://www.abeacha.com/NIST_press_release_bugs_cost.htm</ref>

It is commonly believed that the earlier a defect is found the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found.For example, if a problem in the requirements is found only post-release, then it would cost 10–100 times more to fix than if it had already been found by the requirements review. With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time.

//// Table //////
Another survey by Electric Cloud, the leading provider of software production management (SPM) solutions, conducted in partnership with Osterman Research showed that the majority of software bugs are attributed to poor testing procedures or infrastructure limitations rather than design problems. Additionally, the software test process is generally considered an unpleasant process, with software development professionals rating the use of their companies’ test systems more painful than preparing taxes.
Fifty-eight percent of respondents pointed to problems in the testing process or infrastructure as the cause of their last major bug found in delivered or deployed software, not design defects.

Specifically, the survey highlighted the following:
Completely automated software testing environments are still rare, with just 12 percent of software development organizations using fully automated test systems. Almost 10 percent reported that all testing was done manually.
Forty-six percent of software developers said they do not have time to test as much as they should.
More than a third of developers, 36 percent, said they do not believe their companies perform enough pre-release testing.
Fifty-three percent said their testing is limited by compute resources.<ref>http://www.electric-cloud.com/news/2010-0602.php</ref>
==Importance of Software Testing==
Importance of Software Testing:

The importance of software testing can be explained in brief in the following points.

Error Free Software: Software testing needs to be done thoroughly to deliver an error free software to the client. Even a very minute mistake in the the software can have a disastrous effect for the client. For example a small failure in a banking software can result in a wrong balance and millions of dollars worth loss to the client or customers of the client. So it is of prime importance that the software delivered to the client be bug free and accurate.

Variance from the Requirement:One important factor while building a software is to adhere to the client requirements. if a software is not built in accordance to the requirements then the software becomes useless and redundant for the client.This is a lot of trouble and overhead for the software developing firm also as the requirement was not well understood.The budget fixed for the development of th software can easily go overboard.In such scenarios verification and validation process come into picture. The two most important questions that needs to be answered are “Is the product being built right” and “Is the right product being built”.If the answer is negative to any one of these questions it means that the the product developed has a variance from the client requirements and necessary changes needs to be made before going ahead with further development.

Identify Defect and Prevent their Migration: If a defect is detected in the requirement analysis stage ,then rectifying the defect is a lot easier and cheaper. But if the defect is not identified and carried over to the next phases of software development then it become more difficult to fix the defects,So it is highly recommended that software testing process be started right when the software development starts.

Identify Undiscovered Error: If proper importance is not given to software testing i.e it is not done thoroughly and just a superficial testing is done there is high probability that some of the errors will creep through to the next phase. In such case using different software testing methodologies help in identifying the hidden errors.Exploratory testing is one such method. In such case the tester randomly tests the software for bugs and finds out the error.
Use the Software in Real Time Environment: Testing a software in development and running it in production are two completely different scenarios.When a developer develops a software he tests it only in a development environment. There is high probability that the software will fail miserably in real time environment.

Do Away with Developer Bias: When a person is designated a tester role his prime responsibility is to test the software. The element of bias is removed. For example when we have a developer testing a software in majority of the cases he is going to be biased towards the software. This is because he has developed the software and its natural human instinct to think one’s product is the best. In such scenario many problems will remain unearthed.

Provide Confidence in the Software: Software testing is also used for asserting the confidence on a developed product. There is a huge difference between being an usable product to operable product. When a software is tested time and over again with a huge degree of success then one can easily approve of the quality of the software developed.
<ref>http://www.cbwc-ontario.org/importance-of-software-testing.html</ref>
==Testing Today==
There are various software development life cycle methodologies available for executing software development projects.Each methodology is unique i.e it is designed for a special purpose and compared to other methodologies has its own advantages and disadvantages.But most methodologies work by dividing the entire life cycle into phases and then sharing tasks across this phases.The common methodologies used for software development and their relationship with respect to testing can be summarized below:

The Waterfall model:
It is one of the most common and earliest structured models for software development.Waterfall models stresses that one should only move to a phase after the completing and perfecting the preceding phase.As a result of this the phases of software development in waterfall model are unique and there is no jumping back and forth between the various stages.

A typical waterfall model consists of the following sequential phases

It consists of the following sequential phases through which the development life cycle progresses:
<ref>http://www.guru99.com/testing-methodology.html</ref>

[[File: Waterfall_model.png]]

* Requirement analysis: In this phase, software requirements are captured in such a fashion that they can be translated into actual use cases for the system. The requirements can be derived from performance goals, use cases, target deployment, etc.
* System design: In this phase,the interacting components that make up the system are identified, The exposed interfaces and the communication between them,key algorithms and sequence of interaction are defined, In the end of this phase an architecture and design review is conducted to ensure that the design conforms to the previously defined requirements.
* Implementation : This phase is also termed as Coding and unit testing. In this phase,coding is done for the modules that build the system.Code can also be reviewed and functionality of each module individually tested.
* Verification: Also termed as Integration and system testing. In this phase,all the modules in the system are integrated together and tested as a single system for all the use cases.The primary emphasis is upon making sure that all the modules meet the requirements.
* Deployment and maintenance. This is the final phase where the software system is deployed into the production environment. Any errors identified in this phase are corrected and functionality is added/modified to the system based on the updated requirements.

Waterfall model has the following advantages:
* The life cycle can be compartmentalized into various phases which helps in planning the resources and the amount of effort required through the development process.
* Testing is enforced in every stage in the form of unit testing and reviews.During various stages of the lifecycle different form of reviews like design and code reviews and various forms of testing like unit and integration testing are performed.
* After each phase of lifecycle expectations for deliverables can be set.

Waterfall model has the following disadvantages:
* There is no working version of software until its late in the life cycle.So problems can’t be detcted until the system testing phase.Problems are always hard to fix in the latter phases of life cycle.
* Also for a phase to get started the previous phase must be complete.A system design principle cannot start until the requirement analysis phase is complete and the requirements are frozen. So waterfall model cannot accommodate uncertainties that that may persist after a phase is over.This can lead to delays and extended project schedules.

Testing approach:
During the requirements phase the project requirements are completely defined.Simultaneously the test team brainstorms the scope of testing,test strategy and drafts a detailed test plan.

<h4>Incremental or Iterative Development</h4>

The fundamental principle behind incremental or iterative development is to break down the project into small parts. When one iteration is complete a new module is completed or an existing module is improved. The module is then integrated into the structure and finally the structure is then tested as a whole. In the iterative development model a project is usually broken down into 12 iterations of one to four week duration each.Finally the system is tested at the end of each duration and the test feedback is immediately incorporated. Time spent on the successive iterations can be modified on the basis of experience gained from past iterations. So the system grows by incorporating new functions during the development portion of each iteration.Each iteration thus involves adding of new functions during the development phase.testing evolves as the system evolves.
<br>
<br>
[[File:Iterative_dev.jpg]]
<br>
<br>
The main advantage of iterative development model is that corrective actions can be taken at end of each iteration.

The main disadvantages of iterative development model are:
* Each iteraton involved giving feedback about the deliverables,timelines,efforts and so on.SO the overhead is considerably higher.
* It is hard to freeze the requirements as requirements may need change based on feedback and increasing customer demands.This can lead to more number of iterations and thus delay in deliverables.
* An efficient control change mechanism is needed to manage the system changes made during each iteration.

Testing approach:
As soon as iteration is complete the entire system is subjected to testing.The feedback from testing is immediately available and further incorporated into the next cycle.Testing time required for the successive iterations can be reduced based on the experience gained from past iterations.

<h4>Agile Methodology</h4>
Previously Majority of the software development life cycle methodologies could be categorised into either iterative or sequential model like waterfall model does.But as software systems evolved and became more complex both of these models couldn’t efficiently adapt to the significant and continuous number of changes.Agile methodology was developed to solve this issue.It was develoepd to respond to changes quicly and smoothly. The drawback with iterative model was that even though it removed the disadvantage of sequential models it was still based on the waterfall model.In Agile methodology , software is developed in incremental, rapid cycles. Interactions amongst customers, developers and client are emphasized rather than processes and tools. Agile methodology focuses on responding to change rather than extensive planning.
<br>
<br>
[[File: Agile_dev.gif]]
<br>
<br>
The main differences between agile and traditional methodologies are as follows:
* An incremental method of development is followed rather than the traditional sequential method.There are small incremental releases and each release is based on previous functionality.
* Each release is thoroughly tested and that helps to ensure that the defects are addressed in the next iteration.
* There is more emphasis given on people and their interactions rather than processes and tools.The developers,customers and testers continuously interact with each other.This interaction ensures that the tester us aware of the features being developed during a particular iteration and so can easily identify any sort of discrepancy between the system and the requirements.
* More priority is given to working software rather than detailed documentation. Agile methodologies rely on face-to-face communication and collaboration, with people working in pairs. As there is extensive communication between customers and team members, there is no need for comprehensive requirements document. All agile projects have customers as an integral part of the team.When developers have a query regarding program requirements thy can get it immediately clarified from the customers.

The disadvantage is that a constant client interaction leads to and added time pressure on all stakeholders including the client themselves , software development and test teams .

Testing approach:
Incremental testing approach is followed and thus every release of the project is tested thoroughly.This ensures that any bugs in the system are fixed before the next release.

<h4>Extreme Programming</h4>
Extreme programming<ref>http://xprogramming.com/book/whatisxp</ref> is a form of agile methodology that believes in short development cycles.So rather than designing the whole of the system at the start of the project the preliminary design work is shortened down to solve the simple tasks that have already been identified.The developers have to interact frequently with customers and other developers. A simple task is started and as soon as it is developed customer feedback is taken.The system is delivered to the customer as soon as possible and the requirements are then refined on the basis of customer feedback.So the requirements evolve over a period of time and developers are able to respond to changes quickly.Extreme programming emphasizes on pair programming.This means one developer writes the code for a particular feature and the other developer reviews it. In theory, the driver focuses on the code at hand: the syntax, semantics, and algorithm. The navigator focuses less on that, and more on a level of abstraction higher: the test they are trying to get to pass, the technical task to be delivered next, the time elapsed since all the tests were run, the time elapsed since the last repository commit, and the quality of the overall design. The theory is that pairing results in better designs, fewer bugs, and much better spread of knowledge across a development team, and therefore more functionality per unit time, measured over the long term.
[[File:Extreme_prog.jpg]]

<br>
Extreme Programming is highly useful in the many situations.

* If a customer doesn't have a clear understanding of the system then the developers can interact continuously with the customer ,deliver small pieces and ask the customer for feedback.Corrective action is then taken.
* If a technology used to develop a system is significantly new and its a completely new platform then frequent test cycles in extreme programing mitigate the risk of incompatibility with other existing systems.
* If you want automated unit and functional tests there may be a need to change system design such that each module can be tested in isolation sing automation. XP(Extreme programming) comes handy in such scenario.

The main advantage of following XP is that customers having a vague software design in mind can go ahead to implement their product. The continuous testing and integration ensures that the software code delivered is of the highest standards.

Testing approach:
Extreme programming follows a test driven development.It is explained in brief in the subsequent section.

==Test Driven Development (TDD)==
Test-driven development is one of the core practices of Extreme Programming. Test cases are written first and then code is written to pass the existing test cases. Then new test cases are added to test the existing functionality, and then the entire test suite is run to ensure that the code fails. Then new functionality is added or existing functionality is modified so that the code can withstand the failed test cases. This cycle continues until the test code passes all of the test cases that the team can create. The code is then refactored to make it DRY and more maintainable.

Test-driven development is totally non conventional in the sense that instead of writing code first and then testing it, you write the tests first and then write code to make the tests pass. This is done iteratively. Only when one test case passes, the developer moves on to the next test case and writes code to make it pass. This process is continued until all tests pass.

With test-driven development we can even start with an unclear set of requirements and then can discuss with the customer later about new requirements or change in existing requirements. Even if the code is not ready for a particular functionality but the tests are written, it will ensure that the functionality is addressing all the requirements given by the customer and unnecessary functionality is not included. It allows you to build your software step-by-step and then as the requirements become more clear it evolves the system.<br>
Other advantages of TDD:
* Loosely coupled and highly cohesive code is promoted by Test-driven development because the functionality is evolved in small steps.
* The tests that we write can act as documentation for the final system’s specifications.
* Time for retesting is reduced because of automated testing. So we don’t have to waste time in retesting existing functionality.
* You know exactly what you have to do in order to make a test pass. Your effort can be measured in terms of number of tests passed.
Unit testing is also ensured through Test-driven development. We will still need to all other kinds of testing such as acceptance testing, system integration testing etc.

<h4>Steps in Test-Driven Development</h4>
The following figure shows the steps involved in test-driven development process
<ref>http://msdn.microsoft.com/en-us/library/ff649520.aspx</ref>
<br>
[[File:TDD.gif]]
<br>
==Behavior Driven Development (BDD)==
Behavior Driven Development provides a common language between developers, analysts, and customers, thereby, reducing miscommunication between which was common in earlier forms of software development. This language is used by developers to communicate while developing and explaining code. It makes sure that the customers and the developers have a common vocabulary while talking about the system being developed.

While TDD makes sure that the technical quality of software is up to the mark, behavior-driven development makes sure that the needs of the customer are fulfilled. TDD takes care of the verification part i.e. building the thing right, BDD takes care of the validation part i.e. building the right thing.

Building the Right Thing :

BDD helps to ensure that the right features are built and delivered the first time. By remembering the three categories of problems that we’re typically trying to solve, and by beginning with the stakeholders—the people who are actually going to be using the software we write—we are able to clearly specify what the most important features are, and arrive at a definition of done that encapsulates the business driver for the software.

Reducing Risk :
BDD also reduces risk—risk that, as developers, we’ll go off at a tangent. If our focus is on making a test pass, and that test encapsulates the customer requirement in terms of the behavior of the end result, the likelihood that we’ll get distracted or write something unnecessary is greatly reduced. Interestingly, a suite of acceptance tests developed this way, in partnership with the stakeholder, also forms an excellent starting point for monitoring the system throughout its lifecycle. We know how the system should behave, and if we can automate tests that prove the system is working according to specification, and put alerts around them (both in the development process so we capture defects, and when live so we can resolve and respond to service degradation), we have grounded our monitoring in the behavior of the application that the stakeholder has defined as being of paramount importance to the business.

Evolving Design :
It also helps us to think about the design of the system. The benefits of writing unit tests to increase confidence in our code are pretty obvious. Maturing to the point that we write these tests first helps us focus on writing only the code that is explicitly needed. The tests also serve as a map to the code, and offer lightweight documentation. By tweaking our approach towards thinking about specifying behavior rather than testing classes and methods, we come to appreciate test-driven development as a practice that helps us discover how the system should work, and molds our thinking towards elegant solutions that meet the requirements.

How does all of this relate to Infrastructure as Code? Well, as infrastructure developers, we are providing the underlying systems which make it possible to effectively deliver software. This means our customers are often application developers or test and QA teams. Of course, our customers are also the end users of the software that runs on our systems, so we’re responsible for ensuring our infrastructure performs well and remains available when needed. Having accepted that we need some kind of mechanism for testing our infrastructure to ensure it evolves rapidly without unwanted side effects, bringing the principle of BDD into the equation helps us to ensure that we’re delivering business value by providing the infrastructure that is actually needed. We can avoid wasting time pursuing the latest and greatest technology by realizing we could meet the requirements of the business more readily with a simpler and established solution.
<ref>http://my.safaribooksonline.com/book/software-engineering-and-development/software-testing/9781449309718</ref>
<ref>http://bubusdaybook.blogspot.com/2011/08/extreme-programming-in-nutshell.html</ref>

==Cucumber and RSpec==
Using cucumber, we can describe, in plain text, how a software should behave. It executes plain-text functional descriptions as automated tests. Cucumber supports Behavior Driven Development. The tests are written before the code is written and is verified by non technical stakeholders. The production code is then written to make the stories pass.
<ref> https://github.com/cucumber/cucumber/wiki</ref>

Following is an example of a cucumber scenario in the BackChannel app:
<pre>
Feature: User can add post

Scenario: Add a post
Given I am on the posts index page
When I follow “Add new post”
Then I should be on the Create New Post page
When I fill in “Title” with “Fight Club”
And I fill in “Content” with “The things you own, end up owning you”
And I press “Save Changes”
Then I should be on the posts index page
And I should see “Fight Club”
</pre>

There are six steps to behavior driven development using Cucumber and Rspec<ref>http://cukes.info/</ref>
# Describe behavior in plain text using Cucumber
# Write a step definition in ruby using Rspec
# Run the test and it will fail because the code is not been written yet
# Write code to make the step pass
# Run the test again and see the step pass
# Repeat 2-5 until all steps pass in the behavior
<br>
[[File:Cucumber.jpg|Cucumber and RSpec loop]]
<br>
The test driven development loop is also known as Red-Green-Refactor.
<ref>Aramando, Fox (2012). Engineering Long Lasting Software. San Francisco: Strawberry Canyon</ref>
<br>
Red step: Run the test and verify that it fails because you haven’t yet implemented the code.<br>
Green step: Write the simplest possible code that causes this test to pass without breaking any existing tests.<br>
Refactor step: Refactor the code if there is any scope of refactoring.<br>

==Conclusion==
BDD and TDD may seem strange at first but once you use it you realize you have been using these techniques in conventional development also, while doing unit testing. Rather than coding first and then debugging the code to find the problem, TDD is much better of way of developing a system in that you can isolate the problem really easily because you whole is divided into features and specs.

If we write code first and then debug, we end up using the same techniques as TDD but less efficiently and less productively. Using TDD, bugs can be spotted quickly and regression testing is easy because all the testing is automated.

==References==
<references/>

The video of the lecture can be found [https://www.youtube.com/watch?v=Hpg9303P0Ts here].

File:Waterfall model.png

2012-10-27T00:22:21Z

Sbagga: Waterfall Model

Waterfall Model

File:Iterative dev.jpg

2012-10-27T00:22:08Z

Sbagga: Iterative Development

Iterative Development

File:Agile dev.gif

2012-10-27T00:21:51Z

Sbagga: Agile Development

Agile Development

File:Extreme prog.jpg

2012-10-27T00:16:20Z

Sbagga: Extreme Programming

Extreme Programming

File:TDD.gif

2012-10-27T00:07:27Z

Sbagga: Steps in Test Driven Development

Steps in Test Driven Development

File:Cucumber.jpg

2012-10-26T23:58:36Z

Sbagga: Cucumber and RSpec loop

Cucumber and RSpec loop

CSC/ECE 517 Fall 2012/ch2a 2w29 st

2012-10-26T23:49:23Z

Sbagga:

<h2>SaaS - 5.1 - Testing overview</h2>
__TOC__
==Introduction==
This ia a textbook section that covers the online lecture on [https://www.youtube.com/watch?v=Hpg9303P0Ts Testing Overview].
Gone are the days when the developers used to write code and toss it to the QA team for testing. Today’s developers are far more responsible for testing their own code. Testing these days is far more automated in the sense that the tester doesn’t need to manually check the output if its correct or not. In this document, we give the history of software testing and introduce some of the newer methods that are being used today such as Behavior Driven Development and Test Driven Development.
==History==
Brian Kernighan famously quoted that “Debugging is twice as hard as writing the code in the first place.” Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
In other words it meant if you’re as clever as you can be when you write your code, how will you ever debug it.

Another famous computer scientist Dijkstra famously remarked that "testing can be used to show the presence of bugs but never to show their absence”.The fact is that NOTHING, not inspection, not formal proof, not testing, can give 100% certainty of no errors. Yet all these techniques, at some cost, can in fact reduce the errors to whatever level you wish.

In the initial stages i.e when the software industry was in a nascent stage, there was no stress upon software testing. In most of the cases there was just a quick check on the software and then the software was handed over to the client.A thorough testing of application was considered to be huge waste of time and resources.This was basically done so that items in the checklist could just be ticked off and software could be developed and handed over to client as soon as possible.However as time passed the importance of software testing was realized. Now software testing is given as much importance as that of developing a software.The change in trend can be easily explained.

A Study conducted in 2002 by NIST reported that software bugs cost the U.S. economy a loss of atleast 59.5 billion dollars annually.It further reports that more than one third of this cost could have been avoided if better software testing was performed. <ref>http://www.abeacha.com/NIST_press_release_bugs_cost.htm</ref>

It is commonly believed that the earlier a defect is found the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found.For example, if a problem in the requirements is found only post-release, then it would cost 10–100 times more to fix than if it had already been found by the requirements review. With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time.

//// Table //////
Another survey by Electric Cloud, the leading provider of software production management (SPM) solutions, conducted in partnership with Osterman Research showed that the majority of software bugs are attributed to poor testing procedures or infrastructure limitations rather than design problems. Additionally, the software test process is generally considered an unpleasant process, with software development professionals rating the use of their companies’ test systems more painful than preparing taxes.
Fifty-eight percent of respondents pointed to problems in the testing process or infrastructure as the cause of their last major bug found in delivered or deployed software, not design defects.

Specifically, the survey highlighted the following:
Completely automated software testing environments are still rare, with just 12 percent of software development organizations using fully automated test systems. Almost 10 percent reported that all testing was done manually.
Forty-six percent of software developers said they do not have time to test as much as they should.
More than a third of developers, 36 percent, said they do not believe their companies perform enough pre-release testing.
Fifty-three percent said their testing is limited by compute resources.<ref>http://www.electric-cloud.com/news/2010-0602.php</ref>
==Importance of Software Testing==
Importance of Software Testing:

The importance of software testing can be explained in brief in the following points.

Error Free Software: Software testing needs to be done thoroughly to deliver an error free software to the client. Even a very minute mistake in the the software can have a disastrous effect for the client. For example a small failure in a banking software can result in a wrong balance and millions of dollars worth loss to the client or customers of the client. So it is of prime importance that the software delivered to the client be bug free and accurate.

Variance from the Requirement:One important factor while building a software is to adhere to the client requirements. if a software is not built in accordance to the requirements then the software becomes useless and redundant for the client.This is a lot of trouble and overhead for the software developing firm also as the requirement was not well understood.The budget fixed for the development of th software can easily go overboard.In such scenarios verification and validation process come into picture. The two most important questions that needs to be answered are “Is the product being built right” and “Is the right product being built”.If the answer is negative to any one of these questions it means that the the product developed has a variance from the client requirements and necessary changes needs to be made before going ahead with further development.

Identify Defect and Prevent their Migration: If a defect is detected in the requirement analysis stage ,then rectifying the defect is a lot easier and cheaper. But if the defect is not identified and carried over to the next phases of software development then it become more difficult to fix the defects,So it is highly recommended that software testing process be started right when the software development starts.

Identify Undiscovered Error: If proper importance is not given to software testing i.e it is not done thoroughly and just a superficial testing is done there is high probability that some of the errors will creep through to the next phase. In such case using different software testing methodologies help in identifying the hidden errors.Exploratory testing is one such method. In such case the tester randomly tests the software for bugs and finds out the error.
Use the Software in Real Time Environment: Testing a software in development and running it in production are two completely different scenarios.When a developer develops a software he tests it only in a development environment. There is high probability that the software will fail miserably in real time environment.

Do Away with Developer Bias: When a person is designated a tester role his prime responsibility is to test the software. The element of bias is removed. For example when we have a developer testing a software in majority of the cases he is going to be biased towards the software. This is because he has developed the software and its natural human instinct to think one’s product is the best. In such scenario many problems will remain unearthed.

Provide Confidence in the Software: Software testing is also used for asserting the confidence on a developed product. There is a huge difference between being an usable product to operable product. When a software is tested time and over again with a huge degree of success then one can easily approve of the quality of the software developed.
<ref>http://www.cbwc-ontario.org/importance-of-software-testing.html</ref>
==Testing Today==
There are various software development life cycle methodologies available for executing software development projects.Each methodology is unique i.e it is designed for a special purpose and compared to other methodologies has its own advantages and disadvantages.But most methodologies work by dividing the entire life cycle into phases and then sharing tasks across this phases.The common methodologies used for software development and their relationship with respect to testing can be summarized below:

The Waterfall model:
It is one of the most common and earliest structured models for software development.Waterfall models stresses that one should only move to a phase after the completing and perfecting the preceding phase.As a result of this the phases of software development in waterfall model are unique and there is no jumping back and forth between the various stages.

A typical waterfall model consists of the following sequential phases

It consists of the following sequential phases through which the development life cycle progresses:

/// Picture ///

* Requirement analysis: In this phase, software requirements are captured in such a fashion that they can be translated into actual use cases for the system. The requirements can be derived from performance goals, use cases, target deployment, etc.
* System design: In this phase,the interacting components that make up the system are identified, The exposed interfaces and the communication between them,key algorithms and sequence of interaction are defined, In the end of this phase an architecture and design review is conducted to ensure that the design conforms to the previously defined requirements.
* Implementation : This phase is also termed as Coding and unit testing. In this phase,coding is done for the modules that build the system.Code can also be reviewed and functionality of each module individually tested.
* Verification: Also termed as Integration and system testing. In this phase,all the modules in the system are integrated together and tested as a single system for all the use cases.The primary emphasis is upon making sure that all the modules meet the requirements.
* Deployment and maintenance. This is the final phase where the software system is deployed into the production environment. Any errors identified in this phase are corrected and functionality is added/modified to the system based on the updated requirements.

Waterfall model has the following advantages:
* The life cycle can be compartmentalized into various phases which helps in planning the resources and the amount of effort required through the development process.
* Testing is enforced in every stage in the form of unit testing and reviews.During various stages of the lifecycle different form of reviews like design and code reviews and various forms of testing like unit and integration testing are performed.
* After each phase of lifecycle expectations for deliverables can be set.

Waterfall model has the following disadvantages:
* There is no working version of software until its late in the life cycle.So problems can’t be detcted until the system testing phase.Problems are always hard to fix in the latter phases of life cycle.
* Also for a phase to get started the previous phase must be complete.A system design principle cannot start until the requirement analysis phase is complete and the requirements are frozen. So waterfall model cannot accommodate uncertainties that that may persist after a phase is over.This can lead to delays and extended project schedules.

Testing approach:
During the requirements phase the project requirements are completely defined.Simultaneously the test team brainstorms the scope of testing,test strategy and drafts a detailed test plan.

<h4>Incremental or Iterative Development</h4>

The fundamental principle behind incremental or iterative development is to break down the project into small parts. When one iteration is complete a new module is completed or an existing module is improved. The module is then integrated into the structure and finally the structure is then tested as a whole. In the iterative development model a project is usually broken down into 12 iterations of one to four week duration each.Finally the system is tested at the end of each duration and the test feedback is immediately incorporated. Time spent on the successive iterations can be modified on the basis of experience gained from past iterations. So the system grows by incorporating new functions during the development portion of each iteration.Each iteration thus involves adding of new functions during the development phase.testing evolves as the system evolves.
<br>
/// Image ///
<br>
The main advantage of iterative development model is that corrective actions can be taken at end of each iteration.

The main disadvantages of iterative development model are:
* Each iteraton involved giving feedback about the deliverables,timelines,efforts and so on.SO the overhead is considerably higher.
* It is hard to freeze the requirements as requirements may need change based on feedback and increasing customer demands.This can lead to more number of iterations and thus delay in deliverables.
* An efficient control change mechanism is needed to manage the system changes made during each iteration.

Testing approach:
As soon as iteration is complete the entire system is subjected to testing.The feedback from testing is immediately available and further incorporated into the next cycle.Testing time required for the successive iterations can be reduced based on the experience gained from past iterations.

<h4>Agile Methodology</h4>
Previously Majority of the software development life cycle methodologies could be categorised into either iterative or sequential model like waterfall model does.But as software systems evolved and became more complex both of these models couldn’t efficiently adapt to the significant and continuous number of changes.Agile methodology was developed to solve this issue.It was develoepd to respond to changes quicly and smoothly. The drawback with iterative model was that even though it removed the disadvantage of sequential models it was still based on the waterfall model.In Agile methodology , software is developed in incremental, rapid cycles. Interactions amongst customers, developers and client are emphasized rather than processes and tools. Agile methodology focuses on responding to change rather than extensive planning.
<br>
/// Image ///
<br>
The main differences between agile and traditional methodologies are as follows:
* An incremental method of development is followed rather than the traditional sequential method.There are small incremental releases and each release is based on previous functionality.
* Each release is thoroughly tested and that helps to ensure that the defects are addressed in the next iteration.
* There is more emphasis given on people and their interactions rather than processes and tools.The developers,customers and testers continuously interact with each other.This interaction ensures that the tester us aware of the features being developed during a particular iteration and so can easily identify any sort of discrepancy between the system and the requirements.
* More priority is given to working software rather than detailed documentation. Agile methodologies rely on face-to-face communication and collaboration, with people working in pairs. As there is extensive communication between customers and team members, there is no need for comprehensive requirements document. All agile projects have customers as an integral part of the team.When developers have a query regarding program requirements thy can get it immediately clarified from the customers.

The disadvantage is that a constant client interaction leads to and added time pressure on all stakeholders including the client themselves , software development and test teams .

Testing approach:
Incremental testing approach is followed and thus every release of the project is tested thoroughly.This ensures that any bugs in the system are fixed before the next release.

==BDD and TDD: The Big Picture==
Test-driven development is one of the core practices of Extreme Programming. Test cases are written first and then code is written to pass the existing test cases. Then new test cases are added to test the existing functionality, and then the entire test suite is run to ensure that the code fails. Then new functionality is added or existing functionality is modified so that the code can withstand the failed test cases. This cycle continues until the test code passes all of the test cases that the team can create. The code is then refactored to make it DRY and more maintainable.

Test-driven development is totally non conventional in the sense that instead of writing code first and then testing it, you write the tests first and then write code to make the tests pass. This is done iteratively. Only when one test case passes, the developer moves on to the next test case and writes code to make it pass. This process is continued until all tests pass.

With test-driven development we can even start with an unclear set of requirements and then can discuss with the customer later about new requirements or change in existing requirements. Even if the code is not ready for a particular functionality but the tests are written, it will ensure that the functionality is addressing all the requirements given by the customer and unnecessary functionality is not included. It allows you to build your software step-by-step and then as the requirements become more clear it evolves the system.<br>
Other advantages of TDD:
* Loosely coupled and highly cohesive code is promoted by Test-driven development because the functionality is evolved in small steps.
* The tests that we write can act as documentation for the final system’s specifications.
* Time for retesting is reduced because of automated testing. So we don’t have to waste time in retesting existing functionality.
* You know exactly what you have to do in order to make a test pass. Your effort can be measured in terms of number of tests passed.
Unit testing is also ensured through Test-driven development. We will still need to all other kinds of testing such as acceptance testing, system integration testing etc.

<h4>Steps in Test-Driven Development</h4>
The following figure shows the steps involved in test-driven development process
<br>
/// Image ///
<br>
<h4>Behavior Driven Development</h4>
Behavior Driven Development provides a common language between developers, analysts, and customers, thereby, reducing miscommunication between which was common in earlier forms of software development. This language is used by developers to communicate while developing and explaining code. It makes sure that the customers and the developers have a common vocabulary while talking about the system being developed.

While TDD makes sure that the technical quality of software is up to the mark, behavior-driven development makes sure that the needs of the customer are fulfilled. TDD takes care of the verification part i.e. building the thing right, BDD takes care of the validation part i.e. building the right thing.

Building the Right Thing :

BDD helps to ensure that the right features are built and delivered the first time. By remembering the three categories of problems that we’re typically trying to solve, and by beginning with the stakeholders—the people who are actually going to be using the software we write—we are able to clearly specify what the most important features are, and arrive at a definition of done that encapsulates the business driver for the software.

Reducing Risk :
BDD also reduces risk—risk that, as developers, we’ll go off at a tangent. If our focus is on making a test pass, and that test encapsulates the customer requirement in terms of the behavior of the end result, the likelihood that we’ll get distracted or write something unnecessary is greatly reduced. Interestingly, a suite of acceptance tests developed this way, in partnership with the stakeholder, also forms an excellent starting point for monitoring the system throughout its lifecycle. We know how the system should behave, and if we can automate tests that prove the system is working according to specification, and put alerts around them (both in the development process so we capture defects, and when live so we can resolve and respond to service degradation), we have grounded our monitoring in the behavior of the application that the stakeholder has defined as being of paramount importance to the business.

Evolving Design :
It also helps us to think about the design of the system. The benefits of writing unit tests to increase confidence in our code are pretty obvious. Maturing to the point that we write these tests first helps us focus on writing only the code that is explicitly needed. The tests also serve as a map to the code, and offer lightweight documentation. By tweaking our approach towards thinking about specifying behavior rather than testing classes and methods, we come to appreciate test-driven development as a practice that helps us discover how the system should work, and molds our thinking towards elegant solutions that meet the requirements.

How does all of this relate to Infrastructure as Code? Well, as infrastructure developers, we are providing the underlying systems which make it possible to effectively deliver software. This means our customers are often application developers or test and QA teams. Of course, our customers are also the end users of the software that runs on our systems, so we’re responsible for ensuring our infrastructure performs well and remains available when needed. Having accepted that we need some kind of mechanism for testing our infrastructure to ensure it evolves rapidly without unwanted side effects, bringing the principle of BDD into the equation helps us to ensure that we’re delivering business value by providing the infrastructure that is actually needed. We can avoid wasting time pursuing the latest and greatest technology by realizing we could meet the requirements of the business more readily with a simpler and established solution.
<ref>http://my.safaribooksonline.com/book/software-engineering-and-development/software-testing/9781449309718</ref>
<ref>http://bubusdaybook.blogspot.com/2011/08/extreme-programming-in-nutshell.html</ref>

==Cucumber and RSpec==
Using cucumber, we can describe, in plain text, how a software should behave. It executes plain-text functional descriptions as automated tests. Cucumber supports Behavior Driven Development. The tests are written before the code is written and is verified by non technical stakeholders. The production code is then written to make the stories pass.
<ref> https://github.com/cucumber/cucumber/wiki</ref>

Following is an example of a cucumber scenario in the BackChannel app:
<pre>
Feature: User can add post

Scenario: Add a post
Given I am on the posts index page
When I follow “Add new post”
Then I should be on the Create New Post page
When I fill in “Title” with “Fight Club”
And I fill in “Content” with “The things you own, end up owning you”
And I press “Save Changes”
Then I should be on the posts index page
And I should see “Fight Club”
</pre>

There are six steps to behavior driven development using Cucumber and Rspec<ref>http://cukes.info/</ref>
# Describe behavior in plain text using Cucumber
# Write a step definition in ruby using Rspec
# Run the test and it will fail because the code is not been written yet
# Write code to make the step pass
# Run the test again and see the step pass
# Repeat 2-5 until all steps pass in the behavior
<br>
/// Image ///
<br>
The test driven development loop is also known as Red-Green-Refactor.
<ref>Aramando, Fox (2012). Engineering Long Lasting Software. San Francisco: Strawberry Canyon</ref>
<br>
Red step: Run the test and verify that it fails because you haven’t yet implemented the code.<br>
Green step: Write the simplest possible code that causes this test to pass without breaking any existing tests.<br>
Refactor step: Refactor the code if there is any scope of refactoring.<br>

==Conclusion==
BDD and TDD may seem strange at first but once you use it you realize you have been using these techniques in conventional development also, while doing unit testing. Rather than coding first and then debugging the code to find the problem, TDD is much better of way of developing a system in that you can isolate the problem really easily because you whole is divided into features and specs.

If we write code first and then debug, we end up using the same techniques as TDD but less efficiently and less productively. Using TDD, bugs can be spotted quickly and regression testing is easy because all the testing is automated.

==References==
<references/>

The video of the lecture can be found [https://www.youtube.com/watch?v=Hpg9303P0Ts here].

CSC/ECE 517 Fall 2012

2012-10-15T00:46:09Z

Sbagga:

*[[CSC/ECE 517 Fall 2012/ch1 n xx]]
*[[CSC/ECE 517 Fall 2012/ch1 1w1 rk]]
*[[CSC/ECE 517 Fall 2012/ch1 1w20 pp]]
*[[CSC/ECE 517 Fall 2012/ch1 1w5 su]]
*[[CSC/ECE 517 Fall 2012/ch1 1w6 pp]]
*[[CSC/ECE 517 Fall 2012/ch1 1w4 aj]]
*[[CSC/ECE 517 Fall 2012/ch1 1w7 am]]
*[[CSC/ECE 517 Fall 2012/ch1 1w8 aa]]
*[[CSC/ECE 517 Fall 2012/ch1 1w9 av]]
*[[CSC/ECE 517 Fall 2012/ch1 1w10 pk]]
*[[CSC/ECE 517 Fall 2012/ch1 1w11 ap]]
*[[CSC/ECE 517 Fall 2012/ch1a 1w12 mv]]
*[[CSC/ECE 517 Fall 2012/ch1 1w14 gv]]
*[[CSC/ECE 517 Fall 2012/ch1 1w17 ir]]
*[[CSC/ECE 517 Fall 2012/ch1 1w18 as]]
*[[CSC/ECE 517 Fall 2012/ch1 1w22 an]]
*[[CSC/ECE 517 Fall 2012/ch1 1w21 aa]]
*[[CSC/ECE 517 Fall 2012/ch1 1w21 wi]]
*[[CSC/ECE 517 Fall 2012/ch1 1w31 sa]]
*[[CSC/ECE 517 Fall 2012/ch1a 1w16 br]]
*[[CSC/ECE 517 Fall 2012/ch1a 1w23 as]]
*[[CSC/ECE 517 Fall 2012/ch1 1w24 nr]]
*[[CSC/ECE 517 Fall 2012/ch1 1w15 rt]]
*[[CSC/ECE 517 Fall 2012/ch1 1w3 pl]]
*[[CSC/ECE 517 Fall 2012/ch1 1w32 cm]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w37 ss]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w67 ks]]

*[[CSC/ECE 517 Fall 2012/ch1 1w27 ms]]
*[[CSC/ECE 517 Fall 2012/ch1 1w29 sa]]
*[[CSC/ECE 517 Fall 2012/ch1 1w33 op]]
*[[CSC/ECE 517 Fall 2012/ch1 1w19 sa]]
*[[CSC/ECE 517 Fall 2012/ch1 1w34 vd]]
*[[CSC/ECE 517 Fall 2012/ch1 1w35 sa]]
*[[CSC/ECE 517 Fall 2012/ch1 1w30 rp]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w58 am]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w47 sk]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w69 mv]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w44 as]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w45 is]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w53 kc]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w40 ar]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w39 sn]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w54 go]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w56 ms]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w64 nn]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w66 as]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w40 as]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w42 js]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w46 sm]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w71 gs]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w63 dv]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w55 ms]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w57 mp]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w52 an]]
*[[CSC/ECE_517_Fall_2012/ch1b 1w38 nm]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w60 ac]]
*[[CSC/ECE 517 Fall 2012/ch1b 1w62 rb]]
*[[CSC/ECE 517 Fall 2012/ch2a 2w29 st]]
*[[CSC/ECE 517 Fall 2012/ch 2w30 an]]

CSC/ECE 517 Fall 2012/ch2a 2w29 st

2012-10-15T00:38:46Z

Sbagga: Skeleton

<h2>SaaS - 5.1 - Testing overview</h2>
__TOC__
==Introduction==
This ia a textbook section that covers the online lecture on [https://www.youtube.com/watch?v=Hpg9303P0Ts Testing Overview].

==History==

==Testing Today==
==BDD and TDD: The Big Picture==

==Cucumber and RSpec==

==Conclusion==

==References==
<references/>

The video of the lecture can be found [https://www.youtube.com/watch?v=Hpg9303P0Ts here].

CSC/ECE 517 Fall 2012/ch1b 1w39 sn

2012-10-10T20:08:00Z

Sbagga:

<h2>Lecture 10 - Testing in Rails</h2>
__TOC__
==Introduction==
This article is a summary of [http://mediasite.eos.ncsu.edu/Mediasite/Viewer/?peid=786d700dbc65460695f7f0abf9e8cfa71d Lecture 10] '''"Testing in Rails"'''<ref>http://guides.rubyonrails.org/testing.html</ref> and it basically describes in detail the various types of tests in rails which one might encounter while developing a typical rails application. There are five components central to testing in rails: '''Fixtures''', '''Unit tests''', '''Functional tests''', '''Integration tests''' and '''Performance tests'''. These have been described below.
==Software Testing==
In the simplest terms, software testing can be summarized as follows. We provide some test inputs to the software and we get some test outputs from the software. Then we check if the output is acceptable or not. If the output is acceptable then the test case has passed, otherwise it has failed and we have to debug it. The hard part of doing software testing is selecting a good set of test inputs and designing good acceptability tests.<ref>http://www.udacity.com/wiki/CS258%20Unit%201?course=cs258#what-is-testing</ref>

But while testing the software, we have to keep the following things in mind. We have to find bugs as early as possible. The earlier we find the bug, the cheaper it is to fix it.
Also, more testing is not always better. We may write a lot of test cases but they still may not cover every functionality of our software.
==Setup test environment in Rails==
<h4>In-Memory Databases</h4>

Since all tests involve a high amount of database interaction, it is highly recommended to install the gem ''''memory_test_fix''''<ref>http://agilewebdevelopment.com/plugins/memory_test_fix</ref> which basically ([http://en.wikipedia.org/wiki/Monkey_patch monkey]) patches all tests in rails. This gem allows your tests to mock up a database within the memory, so that all reads/writes to the database executed by the test (when they run) are done to memory instead of the disk. This helps run all the unit tests a lot faster than what they would, if they were to read/write all their results to files (on the disk). It eliminates file locking issues on the test database when running on Windows. This is not a requirement, but it improves the speed of testing and development which is ultimately desirable. Most importantly it is good for testing because one usually does not need the data after the test is done, but only needs it during the lifetime of the test.

Make the following change to the ''''config/database.yml'''' file:
<pre>
test:
adapter: sqlite3
database: ":memory:"
</pre>
The change is that the ''''database:'''' field has been changed from:
<pre>db/development.sqlite3 to ":memory:"</pre>
This now ensures that for all the tests, the database used will be the one in memory and not in an actual Sqlite database.

<h4>Database Setup</h4>
Rails provides a basic boiler plate to create tests.There are three environments provided by Rails - production,development and testing.As the names suggest they are used for different purposes.This prevents developers from messing with their development environments.Inside the rails app directory there will be a directory called test.This directory contains folders-unit,functional,integration and fixtures.The unit folder holds tests for the models, the functional folder is meant to hold tests for your controllers, and the integration folder contains tests that involve any number of controllers interacting.Fixtures contain the sample test data.Rails has the Test::Unit included by default but there are other frameworks also available like RSpec<ref>http://rspec.info/</ref>,Cucumber(for behavior driven development),Shoulda <ref>https://github.com/thoughtbot/shoulda#readme</ref>.When we create the rails scaffold for a particular model then it creates the directories unit,functional,integration which contains the different tests for the respective models.After the test cases have been written we need to prepare the test db.
<pre>
rake db:migrate
rake db:test:load
</pre>
This two commands should suffice but a complete reference of rake commands for testing purpose is mentioned in <ref>http://guides.rubyonrails.org/testing.html#preparing-your-application-for-testing</ref>
After preparing everything we are now ready to run our test.If you are using a Integrated Development Environment(IDE) like RubyMine then you need not worry anything and just do right click on the unit test folder->Select Run->All tests in unit.The figure provided below presents a better picture[[File:RunningTest.png]].

If you are using command line then you can use the following options
<pre>
ruby -Itest test/unit/post_test.rb
Loaded suite unit/post_test
Started
.
Finished in 0.023513 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
</pre>

==Fixtures==
Rails tests are data-driven, which means that all of its tests need some sort of sample data to run on. Fixtures<ref>http://ar.rubyonrails.org/classes/Fixtures.html</ref> allow the tester to populate the testing database before any of the tests in the test folder can run. Fixtures have a file format which describes data structures in a human readable format and can be found under the 'test/fixtures' directory. When the rails [http://guides.rubyonrails.org/getting_started.html#generating-a-model generate model] is executed to create a new model, fixture stubs are automatically created and placed in that directory. YAML fixtures are stored in a single file per model i.e. for every model there is a corresponding fixture. Each record is given a name and is followed by an indented list of key/value pairs in the '''"key: value"''' format. When you create a fixture, it generates an internal hash table. Fixtures are hash objects which can be accessed directly because it is automatically setup as a local variable for the test case. The good thing about this is that we can reference these objects using symbolic names. So if we were to declare a fixture called '''':cookie'''' (see example below), we could reference the entire cookie record simply by:
<pre>categories(:cookie)</pre>
This will return the hash for the fixture named cookie which corresponds to a row in the recipe table describing the recipe for that cookie.

On creating the model, the default fixtures generated are of the form:
<pre>
one:
title: MyString
description: MyString
instructions: MyText

two:
title: MyString
description: MyString
instructions: MyText
</pre>

We spoke of the :cookie fixture which would be defined as:
<pre>
cookie:
Title: Biscuit
Description: Round and Small
Instructions: Buy and bake them
</pre>
This allows us to access this entire record using the symbolic name ':cookie' which hashes to this particular fixture.

An important feature of YAML fixtures is that it supports Embedded Ruby i.e. we can embed ruby code into fixtures to generate a large set of sample data. For example:
<pre>
<% (1..1000).each do |i| %>
fix_<%= i %>:
name: category_<%= i %>
<% end %>
</pre>
This would create a thousand fixtures having symbolic names fix_1, fix_2 up to fix_1000, each one of them having a corresponding name attribute category_1, category_2 etc. This is a much better alternative than having to copy-paste the fixture fixture a thousand times.

A very important thing to remember about fixtures is that the ones which are generated by default by the scaffolds do not factor in for any foreign-key relationships that might be present in the models. Thus, such references have to be explicitly added to the fixture manually in order to reflect any 'has-many' or 'belongs-to' relationships across models.

==Unit Testing==
If the application was created using the scaffold command then it should create a stub in test/unit directory.The initial code would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase
# Replace this with your real tests.
test "the truth" do
assert true
end
end
</pre>
Now if we wanted to add real tests to it then let us take two scenarios
1.Post with empty entries.
2.Post with actual entries
The code for these two test cases would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase

test "Post new empty" do
p = Post.new
assert !p.save, "Saved post without title, content, user, or category"
assert p.invalid?
end

test "Post new correct" do
p = Post.new
#Post has following fields title,email,content
p.title = 'General title'
p.content = 'A new content'
p.email = 'Azrael@ncsu.edu'
#place an assert .so as to find out whether this statement is valid or not
assert p.valid?
end

end

</pre>
test_helper.rb contains the default configuration to run the tests,ActiveSupport::TestCase defines the basic methods for defining
a test case.The test cases must begin with the name "test". The statement that actually determines whether the test has passed or not is the assert statement.An assertion is a line of code that evaluates an object (or expression) for expected results.It can check a variety of things like is the expression true or false,is it valid etc. In this example, in the first test case we are checking whether p is an invalid object,if yes then the test has passed because that is the expected thing.Whereas the second test checks whether p is an valid object or not,if its not then the test fails as the expected output in this case is that p should be a valid object.

Here is another example where we try to test the functionality where a user tries to register with an already existing username
<pre>
test "username exists" do
user = User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
user.save
user1 =User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
assert_false user1.save
end
</pre>
Here, a user tries to add a post with a valid title but he leaves the content field blank.
<pre>
test "empty content test" do
post = Post.new( :title => "No content for this post",:content => nil )
post.User_id=1;
assert_false post.save
end
</pre>
==Functional Testing==
If unit tests covered models then functional tests took care of the controllers.The basic purpose of writing functional tests is to check if all the methods of a controller are working correctly. Since the controllers often influence the content of the web page (which is rendered by the corresponding method of a controller) functional tests are typically written to check if the controller’s method is rendering/redirecting to the correct page, whether or not the users are getting authenticated correctly, validating the correctness of the content displayed on the page,etc.Lets say we have a application where users are allowed to post and then comment on those posts.After the user has made a comment then he has to get redirected to that particular post page.Here is how the create method of the comment controller looks like
<pre>
def create
#@comment = Comment.new(params[:comment])
if(session[:email] == nil)
redirect_to :root
return
end
@comment = Comment.new
@comment.post_id = params[:id]
@comment.content = params[:content_new]
@comment.email = session[:email]
@comment.vote_count = 0

@post = Post.find(@comment.post_id)

dateTime = Time.new
timestamp = dateTime.to_time
@post.update_attributes(:updated_at => timestamp)

respond_to do |format|
if @comment.save
format.html { redirect_to :back }
format.json { render json: @comment, status: :created, location: @comment }
else
format.html { render action: "new" }
format.json { render json: @comment.errors, status: :unprocessable_entity }
end
end
end
</pre>
As it can be seen if no there is no session then no one can comment.If a user is successfully able to comment then he is redirected to the specific post page for which the comment was made.The functional test for this piece of code would look like this<ref>http://guides.rubyonrails.org/testing.html#a-fuller-functional-test-example</ref>
<pre>
class CommentsControllerTest < ActionController::TestCase
setup do
@comment_new = Comment.new(:content => "Comment to create", :email => "test@gm.com", :post_id => 1)
@post = Post.find(@comment_new.post_id)
@comment = comments(:one)#The fixtures contain a row named one
end

test "should create comment" do
assert_difference('Comment.count') do
post :create, { content: @comment_new.content, email: @comment_new.email, post_id: @comment_new.post_id } #parameters that goes with the post request
end

assert_redirected_to post_path(assigns(:post))
end
end
</pre>
As we see from the code that it is important to set the session variable and also we need to know before hand for which post are we commenting so we set those variables in the setup method itself.Inside the test method we attempt to create a new comment and after that we check in the assert statement whether it has been redirected to the correct path which in this case is the post page for which the comment has been made.

Testing the response to your request by asserting the presence of key HTML elements and their content is a useful way to test the views of your application. The assert_select assertion allows you to do this by using a simple yet powerful syntax.

In the example below, we test the functionality upon deleting a user.
<pre>
test "should destroy user" do
assert_difference('User.count', -1) do
delete :destroy, id: @user
end
assert_redirected_to users_path
end
</pre>

==Integration Tests==
Typically in software development, different modules of a project are worked on by different teams/developers. Each team might ensure that the model works correctly in-itself, but this might not necessarily be the case when all the modules are coupled together as a single unit. This is where Integration tests come into play. They test the interaction between multiple controllers and all the components in a sequence, end-to-end. An example of it would be that of a shopping cart application. Even though different phases of the application may work correctly, while running integration tests, one might realize that the ''''add to cart'''' button is absent in the product-catalog, even though the add to cart functionality has been correctly implemented.

The default integration tests framework included in Test-Unit are very low level i.e. they deal with HTTP GET, POST requests responses, session objects, cookies, redirects etc. In ''Behavioral-Driven-Development'' we want to deal with the system on a higher level – similar to a user’s interaction with the system i.e. we want to deal only with clicks, with typing etc. Hence, we can use some of the popular Integration Testing frameworks like Capybara which is a GUI testing framework and allows one to specify - within a test - various actions like 'click' to click on a button, 'fill_in' to fill some text into a designated text-box etc. We can see that this is at a high level and somewhat analogous to actions an end-user might go through while using the application. So the rule of thumb while writing integration tests is to identify the end-users requirements and scope of interaction with the system, walk through the steps that they would take and mimic those in the form of tests. It is clearly evident how such [http://en.wikipedia.org/wiki/Behavior_Driven_Development Behavioral-Driven-Development] goes hand in hand with [http://en.wikipedia.org/wiki/Test-driven_development Test-Driven-Development] and helps in removing the ambiguities which are often associated with Customer Requirements.

The following example shows how the test framework '''CapyBara'''<ref>http://opinionated-programmer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/</ref> is used for Integration Testing:
<pre>
test “create category from main page” do
visit categories_path
click_link “New category”
fill_in “category_name”, :with => “Sample Category”
click_button “Create Category”
end
</pre>
Here we have simulated a user-action (for the CookBook example) where the user would carry out the following steps:
# Visit the Categories Home Page (whose url is specified as categories_path by the routes.rb file)
# Click on the Link which says "New Category", which would lead to another page.
# On this new page, fill the text-box with some text, say "Sample Category"
# Click on the button that says "Create Category".
One can easily identify these actions from the code which is highly intuitive and self-explanatory. Capybara thus provides us with these convenient methods which greatly expedites the whole Integration Testing process.

To use the framework, simply include the corresponding gem in the Gemfile, and the following lines to the end of the ''test_helper.rb'' file.
<pre>
# Add more helper methods ...
require ‘capybara/rails’

class ActionDispatch::IntegrationTest
include Capybara::DSL
end
</pre>
The easy-to-use commands mentioned before are created by Capybara using a Domain Specific Language (DSL) and in order to be able to use it, every Integration test written must '''''require 'test_helper' '''''. This is basically a '''''mixin''''', so one still has the capability to access all the low-level GET/POST commands in Test-Unit in addition to all the methods offered by Capybara.

Another example of Integration Testing with Test-Framework '''RSpec Version 1.3.2'''<ref>https://github.com/rspec/rspec#readme</ref>:
<pre>
describe "Recipes" do

before(:all) do
@recipe = Recipe.new
end

it "should not accept empty recipe" do
@user.should_not_be_valid
@user.title = "Cookie"
@user.description = "Chocolate Chip Cookie"
@user.instructions = "Bake in Oven"
@user.category = 3
@user.should_be_valid
end

end
</pre>
In the above example we basically test that an empty recipe is invalid and that a recipe with fields filled out is valid. This is a very primitive example of using RSpec and it is just to showcase the difference between RSpec and Capybara, and is in no way a comprehensive example. A thing to note is the ''before(:all)'' method, which is similar to the setup() method in Java's JUnit Framework i.e. this method is called before every test in the ''describe'' block gets executed.

In sum, Integration tests are vital and are carried out in the final stages of testing to ensure that the system works as a cohesive and complete unit.

==Performance Tests==
Performance tests as the name indicates are used to gauge the performance of the system and play a very important role in software development for the simple reason that as a developer, one does not want the end user to have a poor experience while using the application. Users do not want to wait long for pages to load and elements on the page to respond. They are not - and should not - be concerned with the capability of the system to handle large loads, scale to accommodate increased volumes of traffic etc. Such details are abstracted away from the user, but they ''do'' have a significant impact on user's interaction with the system.

Rails Performance test can be categorized as a special type of integration tests, which are designed for bench-marking and profiling the test code<ref>http://guides.rubyonrails.org/performance_testing.html#modes</ref>. In these tests, one can mention how many connections are to be simulated to the server etc. at the outcome of which it would be possible to identify the performance bottlenecks and hopefully pinpoint the source of speed and/or memory problems.

Detailed examples can be found [http://guides.rubyonrails.org/performance_testing.html#examples Here].

==Conclusion==
Testing is an indispensable and an inevitable part of development in rails.It must be fully exploited to avail the benefits associated with Test-Driven-Development, for the simple reason that rails provides an excellent in-built framework upon which writing tests is a highly natural and intuitive process. There are many advantages to testing and many articles<ref>http://www.learn.geekinterview.com/programming/ruby/ruby-on-rails-application-testing.html</ref> have been written that emphasize this point.

==References==
<references/>

Most of the content for this article has been obtained from the Lecture taught in class which has been the primary resource. The video of the lecture can be found [http://mediasite.eos.ncsu.edu/Mediasite/Viewer/?peid=786d700dbc65460695f7f0abf9e8cfa71d here].

The old wiki can be found [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_517_Fall_2011/ch4_4e_gs here].

CSC/ECE 517 Fall 2012/ch1b 1w39 sn

2012-10-10T20:05:11Z

Sbagga: Added two more references

<h2>Lecture 10 - Testing in Rails</h2>
__TOC__
==Introduction==
This article is a summary of [http://mediasite.eos.ncsu.edu/Mediasite/Viewer/?peid=786d700dbc65460695f7f0abf9e8cfa71d Lecture 10] '''"Testing in Rails"'''<ref>http://guides.rubyonrails.org/testing.html</ref> and it basically describes in detail the various types of tests in rails which one might encounter while developing a typical rails application. There are five components central to testing in rails: '''Fixtures''', '''Unit tests''', '''Functional tests''', '''Integration tests''' and '''Performance tests'''. These have been described below.
==Software Testing==
In the simplest terms, software testing can be summarized as follows. We provide some test inputs to the software and we get some test outputs from the software. Then we check if the output is acceptable or not. If the output is acceptable then the test case has passed, otherwise it has failed and we have to debug it. The hard part of doing software testing is selecting a good set of test inputs and designing good acceptability tests.<ref>http://www.udacity.com/wiki/CS258%20Unit%201?course=cs258#what-is-testing</ref>

But while testing the software, we have to keep the following things in mind. We have to find bugs as early as possible. The earlier we find the bug, the cheaper it is to fix it.
Also, more testing is not always better. We may write a lot of test cases but they still may not cover every functionality of our software.
==Setup test environment in Rails==
<h4>In-Memory Databases</h4>

Since all tests involve a high amount of database interaction, it is highly recommended to install the gem ''''memory_test_fix''''<ref>http://agilewebdevelopment.com/plugins/memory_test_fix</ref> which basically ([http://en.wikipedia.org/wiki/Monkey_patch monkey]) patches all tests in rails. This gem allows your tests to mock up a database within the memory, so that all reads/writes to the database executed by the test (when they run) are done to memory instead of the disk. This helps run all the unit tests a lot faster than what they would, if they were to read/write all their results to files (on the disk). It eliminates file locking issues on the test database when running on Windows. This is not a requirement, but it improves the speed of testing and development which is ultimately desirable. Most importantly it is good for testing because one usually does not need the data after the test is done, but only needs it during the lifetime of the test.

Make the following change to the ''''config/database.yml'''' file:
<pre>
test:
adapter: sqlite3
database: ":memory:"
</pre>
The change is that the ''''database:'''' field has been changed from:
<pre>db/development.sqlite3 to ":memory:"</pre>
This now ensures that for all the tests, the database used will be the one in memory and not in an actual Sqlite database.

<h4>Database Setup</h4>
Rails provides a basic boiler plate to create tests.There are three environments provided by Rails - production,development and testing.As the names suggest they are used for different purposes.This prevents developers from messing with their development environments.Inside the rails app directory there will be a directory called test.This directory contains folders-unit,functional,integration and fixtures.The unit folder holds tests for the models, the functional folder is meant to hold tests for your controllers, and the integration folder contains tests that involve any number of controllers interacting.Fixtures contain the sample test data.Rails has the Test::Unit included by default but there are other frameworks also available like RSpec<ref>http://rspec.info/</ref>,Cucumber(for behavior driven development),Shoulda <ref>https://github.com/thoughtbot/shoulda#readme</ref>.When we create the rails scaffold for a particular model then it creates the directories unit,functional,integration which contains the different tests for the respective models.After the test cases have been written we need to prepare the test db.
<pre>
rake db:migrate
rake db:test:load
</pre>
This two commands should suffice but a complete reference of rake commands for testing purpose is mentioned in <ref>http://guides.rubyonrails.org/testing.html#preparing-your-application-for-testing</ref>
After preparing everything we are now ready to run our test.If you are using a Integrated Development Environment(IDE) like RubyMine then you need not worry anything and just do right click on the unit test folder->Select Run->All tests in unit.The figure provided below presents a better picture[[File:RunningTest.png]].

If you are using command line then you can use the following options
<pre>
ruby -Itest test/unit/post_test.rb
Loaded suite unit/post_test
Started
.
Finished in 0.023513 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
</pre>

==Fixtures==
Rails tests are data-driven, which means that all of its tests need some sort of sample data to run on. Fixtures<ref>http://ar.rubyonrails.org/classes/Fixtures.html</ref> allow the tester to populate the testing database before any of the tests in the test folder can run. Fixtures have a file format which describes data structures in a human readable format and can be found under the 'test/fixtures' directory. When the rails [http://guides.rubyonrails.org/getting_started.html#generating-a-model generate model] is executed to create a new model, fixture stubs are automatically created and placed in that directory. YAML fixtures are stored in a single file per model i.e. for every model there is a corresponding fixture. Each record is given a name and is followed by an indented list of key/value pairs in the '''"key: value"''' format. When you create a fixture, it generates an internal hash table. Fixtures are hash objects which can be accessed directly because it is automatically setup as a local variable for the test case. The good thing about this is that we can reference these objects using symbolic names. So if we were to declare a fixture called '''':cookie'''' (see example below), we could reference the entire cookie record simply by:
<pre>categories(:cookie)</pre>
This will return the hash for the fixture named cookie which corresponds to a row in the recipe table describing the recipe for that cookie.

On creating the model, the default fixtures generated are of the form:
<pre>
one:
title: MyString
description: MyString
instructions: MyText

two:
title: MyString
description: MyString
instructions: MyText
</pre>

We spoke of the :cookie fixture which would be defined as:
<pre>
cookie:
Title: Biscuit
Description: Round and Small
Instructions: Buy and bake them
</pre>
This allows us to access this entire record using the symbolic name ':cookie' which hashes to this particular fixture.

An important feature of YAML fixtures is that it supports Embedded Ruby i.e. we can embed ruby code into fixtures to generate a large set of sample data. For example:
<pre>
<% (1..1000).each do |i| %>
fix_<%= i %>:
name: category_<%= i %>
<% end %>
</pre>
This would create a thousand fixtures having symbolic names fix_1, fix_2 up to fix_1000, each one of them having a corresponding name attribute category_1, category_2 etc. This is a much better alternative than having to copy-paste the fixture fixture a thousand times.

A very important thing to remember about fixtures is that the ones which are generated by default by the scaffolds do not factor in for any foreign-key relationships that might be present in the models. Thus, such references have to be explicitly added to the fixture manually in order to reflect any 'has-many' or 'belongs-to' relationships across models.

==Unit Testing==
If the application was created using the scaffold command then it should create a stub in test/unit directory.The initial code would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase
# Replace this with your real tests.
test "the truth" do
assert true
end
end
</pre>
Now if we wanted to add real tests to it then let us take two scenarios
1.Post with empty entries.
2.Post with actual entries
The code for these two test cases would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase

test "Post new empty" do
p = Post.new
assert !p.save, "Saved post without title, content, user, or category"
assert p.invalid?
end

test "Post new correct" do
p = Post.new
#Post has following fields title,email,content
p.title = 'General title'
p.content = 'A new content'
p.email = 'Azrael@ncsu.edu'
#place an assert .so as to find out whether this statement is valid or not
assert p.valid?
end

end

</pre>
test_helper.rb contains the default configuration to run the tests,ActiveSupport::TestCase defines the basic methods for defining
a test case.The test cases must begin with the name "test". The statement that actually determines whether the test has passed or not is the assert statement.An assertion is a line of code that evaluates an object (or expression) for expected results.It can check a variety of things like is the expression true or false,is it valid etc. In this example, in the first test case we are checking whether p is an invalid object,if yes then the test has passed because that is the expected thing.Whereas the second test checks whether p is an valid object or not,if its not then the test fails as the expected output in this case is that p should be a valid object.

Here is another example where we try to test the functionality where a user tries to register with an already existing username
<pre>
test "username exists" do
user = User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
user.save
user1 =User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
assert_false user1.save
end
</pre>
Here, a user tries to add a post with a valid title but he leaves the content field blank.
<pre>
test "empty content test" do
post = Post.new( :title => "No content for this post",:content => nil )
post.User_id=1;
assert_false post.save
end
</pre>
==Functional Testing==
If unit tests covered models then functional tests took care of the controllers.The basic purpose of writing functional tests is to check if all the methods of a controller are working correctly. Since the controllers often influence the content of the web page (which is rendered by the corresponding method of a controller) functional tests are typically written to check if the controller’s method is rendering/redirecting to the correct page, whether or not the users are getting authenticated correctly, validating the correctness of the content displayed on the page,etc.Lets say we have a application where users are allowed to post and then comment on those posts.After the user has made a comment then he has to get redirected to that particular post page.Here is how the create method of the comment controller looks like
<pre>
def create
#@comment = Comment.new(params[:comment])
if(session[:email] == nil)
redirect_to :root
return
end
@comment = Comment.new
@comment.post_id = params[:id]
@comment.content = params[:content_new]
@comment.email = session[:email]
@comment.vote_count = 0

@post = Post.find(@comment.post_id)

dateTime = Time.new
timestamp = dateTime.to_time
@post.update_attributes(:updated_at => timestamp)

respond_to do |format|
if @comment.save
format.html { redirect_to :back }
format.json { render json: @comment, status: :created, location: @comment }
else
format.html { render action: "new" }
format.json { render json: @comment.errors, status: :unprocessable_entity }
end
end
end
</pre>
As it can be seen if no there is no session then no one can comment.If a user is successfully able to comment then he is redirected to the specific post page for which the comment was made.The functional test for this piece of code would look like this
<pre>
class CommentsControllerTest < ActionController::TestCase
setup do
@comment_new = Comment.new(:content => "Comment to create", :email => "test@gm.com", :post_id => 1)
@post = Post.find(@comment_new.post_id)
@comment = comments(:one)#The fixtures contain a row named one
end

test "should create comment" do
assert_difference('Comment.count') do
post :create, { content: @comment_new.content, email: @comment_new.email, post_id: @comment_new.post_id } #parameters that goes with the post request
end

assert_redirected_to post_path(assigns(:post))
end
end
</pre>
As we see from the code that it is important to set the session variable and also we need to know before hand for which post are we commenting so we set those variables in the setup method itself.Inside the test method we attempt to create a new comment and after that we check in the assert statement whether it has been redirected to the correct path which in this case is the post page for which the comment has been made.

Testing the response to your request by asserting the presence of key HTML elements and their content is a useful way to test the views of your application. The assert_select assertion allows you to do this by using a simple yet powerful syntax.

In the example below, we test the functionality upon deleting a user.<ref>http://guides.rubyonrails.org/testing.html#a-fuller-functional-test-example</ref>
<pre>
test "should destroy user" do
assert_difference('User.count', -1) do
delete :destroy, id: @user
end
assert_redirected_to users_path
end
</pre>

==Integration Tests==
Typically in software development, different modules of a project are worked on by different teams/developers. Each team might ensure that the model works correctly in-itself, but this might not necessarily be the case when all the modules are coupled together as a single unit. This is where Integration tests come into play. They test the interaction between multiple controllers and all the components in a sequence, end-to-end. An example of it would be that of a shopping cart application. Even though different phases of the application may work correctly, while running integration tests, one might realize that the ''''add to cart'''' button is absent in the product-catalog, even though the add to cart functionality has been correctly implemented.

The default integration tests framework included in Test-Unit are very low level i.e. they deal with HTTP GET, POST requests responses, session objects, cookies, redirects etc. In ''Behavioral-Driven-Development'' we want to deal with the system on a higher level – similar to a user’s interaction with the system i.e. we want to deal only with clicks, with typing etc. Hence, we can use some of the popular Integration Testing frameworks like Capybara which is a GUI testing framework and allows one to specify - within a test - various actions like 'click' to click on a button, 'fill_in' to fill some text into a designated text-box etc. We can see that this is at a high level and somewhat analogous to actions an end-user might go through while using the application. So the rule of thumb while writing integration tests is to identify the end-users requirements and scope of interaction with the system, walk through the steps that they would take and mimic those in the form of tests. It is clearly evident how such [http://en.wikipedia.org/wiki/Behavior_Driven_Development Behavioral-Driven-Development] goes hand in hand with [http://en.wikipedia.org/wiki/Test-driven_development Test-Driven-Development] and helps in removing the ambiguities which are often associated with Customer Requirements.

The following example shows how the test framework '''CapyBara'''<ref>http://opinionated-programmer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/</ref> is used for Integration Testing:
<pre>
test “create category from main page” do
visit categories_path
click_link “New category”
fill_in “category_name”, :with => “Sample Category”
click_button “Create Category”
end
</pre>
Here we have simulated a user-action (for the CookBook example) where the user would carry out the following steps:
# Visit the Categories Home Page (whose url is specified as categories_path by the routes.rb file)
# Click on the Link which says "New Category", which would lead to another page.
# On this new page, fill the text-box with some text, say "Sample Category"
# Click on the button that says "Create Category".
One can easily identify these actions from the code which is highly intuitive and self-explanatory. Capybara thus provides us with these convenient methods which greatly expedites the whole Integration Testing process.

To use the framework, simply include the corresponding gem in the Gemfile, and the following lines to the end of the ''test_helper.rb'' file.
<pre>
# Add more helper methods ...
require ‘capybara/rails’

class ActionDispatch::IntegrationTest
include Capybara::DSL
end
</pre>
The easy-to-use commands mentioned before are created by Capybara using a Domain Specific Language (DSL) and in order to be able to use it, every Integration test written must '''''require 'test_helper' '''''. This is basically a '''''mixin''''', so one still has the capability to access all the low-level GET/POST commands in Test-Unit in addition to all the methods offered by Capybara.

Another example of Integration Testing with Test-Framework '''RSpec Version 1.3.2'''<ref>https://github.com/rspec/rspec#readme</ref>:
<pre>
describe "Recipes" do

before(:all) do
@recipe = Recipe.new
end

it "should not accept empty recipe" do
@user.should_not_be_valid
@user.title = "Cookie"
@user.description = "Chocolate Chip Cookie"
@user.instructions = "Bake in Oven"
@user.category = 3
@user.should_be_valid
end

end
</pre>
In the above example we basically test that an empty recipe is invalid and that a recipe with fields filled out is valid. This is a very primitive example of using RSpec and it is just to showcase the difference between RSpec and Capybara, and is in no way a comprehensive example. A thing to note is the ''before(:all)'' method, which is similar to the setup() method in Java's JUnit Framework i.e. this method is called before every test in the ''describe'' block gets executed.

In sum, Integration tests are vital and are carried out in the final stages of testing to ensure that the system works as a cohesive and complete unit.

==Performance Tests==
Performance tests as the name indicates are used to gauge the performance of the system and play a very important role in software development for the simple reason that as a developer, one does not want the end user to have a poor experience while using the application. Users do not want to wait long for pages to load and elements on the page to respond. They are not - and should not - be concerned with the capability of the system to handle large loads, scale to accommodate increased volumes of traffic etc. Such details are abstracted away from the user, but they ''do'' have a significant impact on user's interaction with the system.

Rails Performance test can be categorized as a special type of integration tests, which are designed for bench-marking and profiling the test code<ref>http://guides.rubyonrails.org/performance_testing.html#modes</ref>. In these tests, one can mention how many connections are to be simulated to the server etc. at the outcome of which it would be possible to identify the performance bottlenecks and hopefully pinpoint the source of speed and/or memory problems.

Detailed examples can be found [http://guides.rubyonrails.org/performance_testing.html#examples Here].

==Conclusion==
Testing is an indispensable and an inevitable part of development in rails.It must be fully exploited to avail the benefits associated with Test-Driven-Development, for the simple reason that rails provides an excellent in-built framework upon which writing tests is a highly natural and intuitive process. There are many advantages to testing and many articles<ref>http://www.learn.geekinterview.com/programming/ruby/ruby-on-rails-application-testing.html</ref> have been written that emphasize this point.

==References==
<references/>

Most of the content for this article has been obtained from the Lecture taught in class which has been the primary resource. The video of the lecture can be found [http://mediasite.eos.ncsu.edu/Mediasite/Viewer/?peid=786d700dbc65460695f7f0abf9e8cfa71d here].

The old wiki can be found [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_517_Fall_2011/ch4_4e_gs here].

CSC/ECE 517 Fall 2012/ch1b 1w39 sn

2012-10-03T22:38:12Z

Sbagga:

<h2>Lecture 10 - Testing in Rails</h2>
==Software Testing==
In the simplest terms, software testing can be summarized as follows. We provide some test inputs to the software and we get some test outputs from the software. Then we check if the output is acceptable or not. If the output is acceptable then the test case has passed, otherwise it has failed and we have to debug it. The hard part of doing software testing is selecting a good set of test inputs and designing good acceptability tests.

But while testing the software, we have to keep the following things in mind. We have to find bugs as early as possible. The earlier we find the bug, the cheaper it is to fix it.
Also, more testing is not always better. We may write a lot of test cases but they still may not cover every functionality of our software.
==Setup test environment in Rails==
Rails provides a basic boiler plate to create tests.There are three environments provided by Rails - production,development and testing.As the names suggest they are used for different purposes.This prevents developers from messing with their development environments.Inside the rails app directory there will be a directory called test.This directory contains folders-unit,functional,integration and fixtures.The unit folder holds tests for the models, the functional folder is meant to hold tests for your controllers, and the integration folder contains tests that involve any number of controllers interacting.Fixtures contain the sample test data.Rails has the Test::Unit included by default but there are other frameworks also available like RSpec<ref>http://rspec.info/</ref>,Cucumber(for behavior driven development),Shoulda <ref>https://github.com/thoughtbot/shoulda#readme</ref>.When we create the rails scaffold for a particular model then it creates the directories unit,functional,integration which contains the different tests for the respective models.After the test cases have been written we need to prepare the test db.
<pre>
rake db:migrate
rake db:test:load
</pre>
This two commands should suffice but a complete reference of rake commands for testing purpose is mentioned in <ref>http://guides.rubyonrails.org/testing.html#preparing-your-application-for-testing</ref>
After preparing everything we are now ready to run our test.If you are using a Integrated Development Environment(IDE) like RubyMine then you need not worry anything and just do right click on the unit test folder->Select Run->All tests in unit.The figure provided below presents a better picture[[File:RunningTest.png]].

If you are using command line then you can use the following options
<pre>
ruby -Itest test/unit/post_test.rb
Loaded suite unit/post_test
Started
.
Finished in 0.023513 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
</pre>

<h4>Unit Testing:</h4>
If the application was created using the scaffold command then it should create a stub in test/unit directory.The initial code would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase
# Replace this with your real tests.
test "the truth" do
assert true
end
end
</pre>
Now if we wanted to add real tests to it then let us take two scenarios
1.Post with empty entries.
2.Post with actual entries
The code for these two test cases would look something like this
<pre>
require 'test_helper'

class PostTest < ActiveSupport::TestCase

test "Post new empty" do
p = Post.new
assert !p.save, "Saved post without title, content, user, or category"
assert p.invalid?
end

test "Post new correct" do
p = Post.new
#Post has following fields title,email,content
p.title = 'General title'
p.content = 'A new content'
p.email = 'Azrael@ncsu.edu'
#place an assert .so as to find out whether this statement is valid or not
assert p.valid?
end

end

</pre>
test_helper.rb contains the default configuration to run the tests,ActiveSupport::TestCase defines the basic methods for defining
a test case.The test cases must begin with the name "test". The statement that actually determines whether the test has passed or not is the assert statement.An assertion is a line of code that evaluates an object (or expression) for expected results.It can check a variety of things like is the expression true or false,is it valid etc. In this example, in the first test case we are checking whether p is an invalid object,if yes then the test has passed because that is the expected thing.Whereas the second test checks whether p is an valid object or not,if its not then the test fails as the expected output in this case is that p should be a valid object.

Here is another example where we try to test the functionality where a user tries to register with an already existing username
<pre>
test "username exists" do
user = User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
user.save
user1 =User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
assert_false user1.save
end
</pre>
Here, a user tries to add a post with a valid title but he leaves the content field blank.
<pre>
test "empty content test" do
post = Post.new( :title => "No content for this post",:content => nil )
post.User_id=1;
assert_false post.save
end
</pre>
<h4>Functional Testing</h4>
If unit tests covered models then functional tests took care of the controllers.The basic purpose of writing functional tests is to check if all the methods of a controller are working correctly. Since the controllers often influence the content of the web page (which is rendered by the corresponding method of a controller) functional tests are typically written to check if the controller’s method is rendering/redirecting to the correct page, whether or not the users are getting authenticated correctly, validating the correctness of the content displayed on the page,etc.Lets say we have a application where users are allowed to post and then comment on those posts.After the user has made a comment then he has to get redirected to that particular post page.Here is how the create method of the comment controller looks like
<pre>
def create
#@comment = Comment.new(params[:comment])
if(session[:email] == nil)
redirect_to :root
return
end
@comment = Comment.new
@comment.post_id = params[:id]
@comment.content = params[:content_new]
@comment.email = session[:email]
@comment.vote_count = 0

@post = Post.find(@comment.post_id)

dateTime = Time.new
timestamp = dateTime.to_time
@post.update_attributes(:updated_at => timestamp)

respond_to do |format|
if @comment.save
format.html { redirect_to :back }
format.json { render json: @comment, status: :created, location: @comment }
else
format.html { render action: "new" }
format.json { render json: @comment.errors, status: :unprocessable_entity }
end
end
end
</pre>
As it can be seen if no there is no session then no one can comment.If a user is successfully able to comment then he is redirected to the specific post page for which the comment was made.The functional test for this piece of code would look like this
<pre>
class CommentsControllerTest < ActionController::TestCase
setup do
@comment_new = Comment.new(:content => "Comment to create", :email => "test@gm.com", :post_id => 1)
@post = Post.find(@comment_new.post_id)
@comment = comments(:one)#The fixtures contain a row named one
end

test "should create comment" do
assert_difference('Comment.count') do
post :create, { content: @comment_new.content, email: @comment_new.email, post_id: @comment_new.post_id } #parameters that goes with the post request
end

assert_redirected_to post_path(assigns(:post))
end
end
</pre>
As we see from the code that it is important to set the session variable and also we need to know before hand for which post are we commenting so we set those variables in the setup method itself.Inside the test method we attempt to create a new comment and after that we check in the assert statement whether it has been redirected to the correct path which in this case is the post page for which the comment has been made.

Testing the response to your request by asserting the presence of key HTML elements and their content is a useful way to test the views of your application. The assert_select assertion allows you to do this by using a simple yet powerful syntax.

In the example below, we test the functionality upon deleting a user.
<pre>
test "should destroy user" do
assert_difference('User.count', -1) do
delete :destroy, id: @user
end
assert_redirected_to users_path
end
</pre>