User:Ewhorton: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(15 intermediate revisions by 2 users not shown)
Line 11: Line 11:


===Motivation===
===Motivation===
The moment a line of code is written it becomes legacy code in need of support. Test cases are used not only to ensure that code is operating properly, but also to ensure that it continues to do so after having been modified. An important aspect of this is acceptance testing to verify that the project meets customer requirements. Rather than performing these tests by hand, it is easier to combine an automated testing system like RSpec with a driver like Capybara which allows testing of an application's functionality from the outside either by running in a  headless browser.
The moment a line of code is written it becomes legacy code in need of support. Test cases are used not only to ensure that code is operating properly, but also to ensure that it continues to do so after having been modified. An important aspect of this is acceptance testing to verify that the project meets customer requirements. Rather than performing these tests by hand, it is easier to combine an automated testing system like RSpec with a driver like Capybara which allows testing of an application's functionality from the outside by running in a  browser.


===Flow of the calibration function===
===Flow of the calibration function===
Line 17: Line 17:




Before analyzing code, one must familiarize themselves with the steps involved in assignment submission by student on Expertiza. The steps involved are:
Before analyzing code, one must familiarize themselves with the steps involved in calibration on Expertiza. The steps involved are:
# Login with valid instructor username and password
# Login with valid instructor username and password
# Click on "Assignment"
# Click on "Assignment"
Line 46: Line 46:
* factory_girl_rails
* factory_girl_rails


====rspec-rails====
====Rspec-rails====
Rspec-rails is a testing framework for Rails 3.x and 4.x. It supports testing of models, controllers, requests, features, views and routes. It does this by accepting test scenarios called specs.<ref>[https://github.com/rspec/rspec-rails rpsce-rails on GitHub]</ref>
Rspec-rails is a testing framework for Rails 3.x and 4.x. It supports testing of models, controllers, requests, features, views and routes. It does this by accepting test scenarios called specs.<ref>[https://github.com/rspec/rspec-rails rpsce-rails on GitHub]</ref>


====capybara====
====Capybara====
Capybara helps you test web applications by simulating how a real user would interact with your application. It comes with built in Rack::Test and Selenium support. WebKit is supported through an external gem.<ref>[https://github.com/jnicklas/capybara capybara on GitHub]</ref> To control the environments in which the scenarios are run, it provides before and after hooks.<ref>[https://www.relishapp.com/rspec/rspec-core/v/2-2/docs/hooks/before-and-after-hooks Before and after hooks]</ref>
Capybara helps you test web applications by simulating how a real user would interact with your application. It comes with built in Rack::Test and Selenium support. WebKit is supported through an external gem.<ref>[https://github.com/jnicklas/capybara capybara on GitHub]</ref> To control the environments in which the scenarios are run, it provides before and after hooks.<ref>[https://www.relishapp.com/rspec/rspec-core/v/2-2/docs/hooks/before-and-after-hooks Before and after hooks]</ref>
*before(:each) blocks are run before each scenario in the group
*before(:each) blocks are run before each scenario in the group
Line 56: Line 56:
*after(:all) blocks are run once after all of the scenarios in the group
*after(:all) blocks are run once after all of the scenarios in the group


====selenium====
====Selenium====
Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It is the default javascript webdriver for Capybara.
Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It is the default javascript webdriver for Capybara.


====factory girl====
====Factory Girl====
Factory Girl is a factory gem meant for use in testing. It allows for easy and highly configurable on-the-fly model creation that allows tests to request a new instance of a model template and optionally override any of the defaults.
Factory Girl <ref>[https://github.com/thoughtbot/factory_girl Factory Girl]</ref> is a factory gem meant for use in testing. It allows for easy and highly configurable on-the-fly model creation that allows tests to request a new instance of a model template and optionally override any of the defaults.


===Test Scenarios===
===Test Scenarios===
Line 99: Line 99:
:It is currently the review phase for an assignment
:It is currently the review phase for an assignment
:An instructor authenticates to Expertiza and chooses to view calibration
:An instructor authenticates to Expertiza and chooses to view calibration
:The instructor can see the percentages relating to student responses for the assignment
:The instructor can see the student and instructor responses for the assignment
</div>
</div>


===Code===  
===Code===  
Code for the feature test consist in two files:
Code for the tests exists in two locations
* [https://github.com/shrenujgandhi/expertiza/blob/master/spec/assignment_setup.rb assignment_setup.rb] which contains method for creating assignments using controller calls.
* [https://github.com/EricHorton/expertiza/blob/master/spec/factories/factories.rb Factory]
* [https://github.com/shrenujgandhi/expertiza/blob/master/spec/features/student_submission_spec.rb student_assignment_submission.rb] which contain calls to assignment_setup, scenarios to test assignment submission and commands to delete the created assignments.  
* [https://github.com/EricHorton/expertiza/blob/master/spec/features/calibration_spec.rb CalibrationSpec]
 
====Design Pattern====
We attempted to follow good OO practices by implementing the CRUD<ref>[https://en.wikipedia.org/wiki/Create,_read,_update_and_delete Create Read Update and Delete]</ref> design pattern using Factory Girl and Capybara.


====Structure====
====Structure====
Our specs follow the standard RSpec describe/it syntax. For example:


<div style="width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
<div style="width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
<b>require</b> files


<b>RSpec.feature 'assignment submission when student' do</b>


:<b>before(:all) do</b>
describe 'Some Feature' do
::create assignments for submission
:before :each do
:<b>end</b>


:<b>before(:each) do</b>
:end
::capybara steps to login
:<b>end</b>


:<b>after(:all) do</b>
:it 'should do x' do
::delete assignments created
::expect(true).to be(true)
:<b>end</b>
:end


:<b>scenario 'submits only valid link to ongoing assignment' do</b>
:it 'should not do y' do
::mock steps using capybara
::expect(true).to be(false)
::make assertions
:end
:<b>end</b>
end
 
:....
:....
:....
 
<b>end</b>
</div>
</div>
====Sample Scenario====
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
require 'rails_helper'
require 'spec_helper'
require 'assignment_setup'
RSpec.feature 'assignment submission when student' do
  active_assignment="FeatureTest"
  expired_assignment="LibraryRailsApp"
  d = Date.parse(Time.now.to_s)
  due_date1=(d >> 1).strftime("%Y-%m-%d %H:%M:00")
  due_date2=(d << 1).strftime("%Y-%m-%d %H:%M:00")
  # Before all block runs once before all the scenarios are tested
  before(:all) do
    # Create active/ongoing assignment
    create_assignment(active_assignment, due_date1)
    # Create expired/finished assignment
    create_assignment(expired_assignment, due_date2)
  end
  # Before each block runs before every scenario
  before(:each) do
    # Login as a student before each scenario
    visit root_path
    fill_in 'User Name', :with => 'student13'
    fill_in 'Password', :with => 'password'
    click_on 'SIGN IN'
  end
  # After all block runs after all the scenarios are tested
  after(:all)do
    # Delete active/ongoing assignment created by the test
    assignment = Assignment.find_by_name(active_assignment)
    assignment.delete
    # Delete expired/finished assignment created by the test
    assignment = Assignment.find_by_name(expired_assignment)
    assignment.delete
  end
  # Scenario to check whether student is able to submit valid link to an ongoing assignment
  scenario 'submits only valid link to ongoing assignment' do
    click_on active_assignment
    click_on 'Your work'
    fill_in 'submission', :with => 'http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule'
    click_on 'Upload link'
    expect(page).to have_content 'http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule'
  end
end
</pre>


==Running  the tests==
==Running  the tests==
Line 200: Line 137:
* Clone the repository in a new directory
* Clone the repository in a new directory
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
$ mkdir review
$ git clone https://github.com/EricHorton/expertiza
$ cd review
$ git clone https://github.com/shrenujgandhi/expertiza.git
</pre>
</pre>


In case you don't have a database with student and instructor entries then download the dump from https://drive.google.com/a/ncsu.edu/file/d/0B2vDvVjH76uESEkzSWpJRnhGbmc/view. Extract its contents.
Create and then prepare the test database
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
$ cd expertiza
$ cd expertiza
$ rake db:create:all
$ rake db:test:prepare
$ cd Downloads/
$ mysql -u root -p expertiza_development < expertiza_scrubbed_2015_08_14.sql
password:
 
$ cd expertiza
$ rake db:migrate
</pre>
</pre>


* Type the command to run tests for assignment submission
* Type the command to run tests for assignment submission
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
<pre style="font-size:12.5px; width:750px; margin-top:10px; border:1px solid lightgrey; background:ivory; color:black; padding-left:15px; padding-top:15px; padding-bottom:15px;">
rspec spec/features/student_submission_spec.rb  
rspec spec/features/calibration_spec.rb --order default
</pre>
</pre>


Line 226: Line 155:
The following screenshot shows the result of the rspec command.
The following screenshot shows the result of the rspec command.


[[File:test_result_temp2_expertiza.jpg|Feature Test Result]]
[[File:calibration-tests.png|Calibration Tests]]
 
=== Test Analysis ===
=== Test Analysis ===


* ''7 examples, 0 failures'' imply that there are 7 test scenarios and none of them failed.
* All test cases are green, which means there are no currently failing tests.
* The tests take around 35 seconds to run, which implies that they are fast.
* Test cases are run through the Selenium driver on Firefox for javascript support. Because they are full acceptance tests they are slower than regular unit tests, but still beat evaluating the feature by hand.
* They are also repeatable as the assignments that are created in the ''before all'' block are also deleted in the ''after all'' block.
* Tests are independent and repeatable. Database changes are made within a transaction that is rolled back after tests complete.
* They are also self-verifying as all the results are in green, which mean that all the cases passed.


==Project Resources==
==Project Resources==
# [https://github.com/shrenujgandhi/expertiza GitHub Link]
# [https://github.com/EricHorton/expertiza GitHub Fork]
# [https://youtu.be/_58l0VPy04M YouTube Demo]


=References=
=References=

Latest revision as of 22:13, 21 March 2016

Expertiza is a web application where students can submit and peer-review learning objects (articles, code, web sites, etc). It is used in select courses at NC State and by professors at several other colleges and universities. <ref>Expertiza on GitHub</ref>


Introduction

Our contribution to this project is a suite of functional tests for the assignment calibration function. These tests are designed to provide integration, regression, and acceptance testing by using automation technologies.

Problem Statement

Expertiza has a feature called Calibration that can be turned on for an assignment. When this feature is enabled an expert, typically the instructor, can provide an expert review for the assignment artifacts. After students finish their own reviews they may compare their responses to the expert review. The instructor may also view calibration results to view the scores for the entire class.

While this is a preexisting feature within Expertiza, there currently exist no tests for it. The goal of this project is to fully test the entire flow of the calibration feature using the RSpec and Capybara frameworks.

Motivation

The moment a line of code is written it becomes legacy code in need of support. Test cases are used not only to ensure that code is operating properly, but also to ensure that it continues to do so after having been modified. An important aspect of this is acceptance testing to verify that the project meets customer requirements. Rather than performing these tests by hand, it is easier to combine an automated testing system like RSpec with a driver like Capybara which allows testing of an application's functionality from the outside by running in a browser.

Flow of the calibration function

A more detailed description of the calibration may be found here


Before analyzing code, one must familiarize themselves with the steps involved in calibration on Expertiza. The steps involved are:

  1. Login with valid instructor username and password
  2. Click on "Assignment"
  3. Click on "New public assignment"
  4. Input all the information needed, check the “Calibrated peer-review for training?” box, click on Create
  5. Select the review rubric in “Rubrics” tab
  6. Go to “Due dates” tab, make sure that this assignment is currently in Review phase, also, make the submission allowed in review phase.
  7. Add a submitter to the assignment. This submitter will add documents to be reviewed.
  8. Go to the list of assignments, click “edit” icon of the assignment
  9. Go to the “Calibration” tab, click on the "Begin" link next to the student artifact for the expert review, and click “Submit Review”
  10. Go to “Review strategy” tab, select Instructor-Selected as review strategy, then select the radio button “Set number of reviews done by each student” and input the number of reviews
  11. Login with valid student username and password who is a reviewer for the assignment
  12. Click on the assignment
  13. Click on “Other’s work”
  14. After clicking “Begin”, Expertiza will display peer-review page
  15. Click “Submit Review”
  16. Click “Show calibration results” to see expert review
  17. Login with valid instructor username and password
  18. Click on "Assignment"
  19. Click on the “view review report” to see the calibration report

Creating the Tests

Gems involved

  • rspec-rails
  • capybara
  • selenium-webdriver
  • factory_girl_rails

Rspec-rails

Rspec-rails is a testing framework for Rails 3.x and 4.x. It supports testing of models, controllers, requests, features, views and routes. It does this by accepting test scenarios called specs.<ref>rpsce-rails on GitHub</ref>

Capybara

Capybara helps you test web applications by simulating how a real user would interact with your application. It comes with built in Rack::Test and Selenium support. WebKit is supported through an external gem.<ref>capybara on GitHub</ref> To control the environments in which the scenarios are run, it provides before and after hooks.<ref>Before and after hooks</ref>

  • before(:each) blocks are run before each scenario in the group
  • before(:all) blocks are run once before all of the scenarios in the group
  • after(:each) blocks are run after each scenario in the group
  • after(:all) blocks are run once after all of the scenarios in the group

Selenium

Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It is the default javascript webdriver for Capybara.

Factory Girl

Factory Girl <ref>Factory Girl</ref> is a factory gem meant for use in testing. It allows for easy and highly configurable on-the-fly model creation that allows tests to request a new instance of a model template and optionally override any of the defaults.

Test Scenarios

The following scenarios were considered when writing tests for the calibration use case:

An instructor creates a calibrated assignment

An instructor authenticates to expertiza
They select to add a new assignment
While creating the assignment, the instructor selects to turn calibration on
After creating the assignment the instructor sees a calibration tab on the assignment page

An instructor edits the calibration configuration

An instructor authenticates to expertiza
They select to edit an existing assignment
The instructor clicks on the calibration tab and sees it is populated with submitted artifacts

An expert submits a review for calibration

An expert has been added to an assignment to provide the expert review
The expert authenticates to Expertiza and begins the expert review
The expert finishes and submits the review

Student Calibration

A student has submitted a response for the assignment and it is currently in the review phase
The student authenticates to Expertiza and chooses to view calibration for the assignment
The student sees their own answers compared to the expert review.
Answers that match are marked green, and answers that differ are marked red.

Instructor Calibration

It is currently the review phase for an assignment
An instructor authenticates to Expertiza and chooses to view calibration
The instructor can see the student and instructor responses for the assignment

Code

Code for the tests exists in two locations

Design Pattern

We attempted to follow good OO practices by implementing the CRUD<ref>Create Read Update and Delete</ref> design pattern using Factory Girl and Capybara.

Structure

Our specs follow the standard RSpec describe/it syntax. For example:


describe 'Some Feature' do

before :each do
end
it 'should do x' do
expect(true).to be(true)
end
it 'should not do y' do
expect(true).to be(false)
end

end

Running the tests

The following are steps required to run the test

  • Clone the repository in a new directory
$ git clone https://github.com/EricHorton/expertiza

Create and then prepare the test database

$ cd expertiza
$ rake db:test:prepare
  • Type the command to run tests for assignment submission
rspec spec/features/calibration_spec.rb --order default

Test Results

The following screenshot shows the result of the rspec command.

Calibration Tests

Test Analysis

  • All test cases are green, which means there are no currently failing tests.
  • Test cases are run through the Selenium driver on Firefox for javascript support. Because they are full acceptance tests they are slower than regular unit tests, but still beat evaluating the feature by hand.
  • Tests are independent and repeatable. Database changes are made within a transaction that is rolled back after tests complete.

Project Resources

  1. GitHub Fork

References

<references></references>

External links