CSC/ECE 517 Fall 2015/oss E1574 BKS

Expertiza is a web application where students can submit and peer-review learning objects (articles, code, web sites, etc). It is used in select courses at NC State and by professors at several other colleges and universities. <ref>Expertiza on GitHub</ref>

Introduction

Our contribution in this project is to write feature test for assignment submission by student. The test would mock steps taken by the student to manually submit the assignment.

Problem Statement

Once an assignment is created by the instructor the only call to action for the student is assignment submission. Currently, there is no feature test for assignment submission. The goal is to understand the flow of the assignment submission by student manually and mock this flow in the Feature Tests using RSpec and Capybara.

Motivation

Once the application is built, it is necessary to test its functionality. More importantly, it is essential to test this functionality from users perspective. Feature spec allows testing of your application's functionality from the outside either by simulating a browser or by headless browser simulation.

Assumptions

To successfully implement each scenario in the test, the following assumptions are taken:

1. The feature test will use the development environment. In particular, it will use the expertiza_development database.
2. An instructor entry is assumed to be present in the database. The credentials for this instructor is instructor6 as username and password as password.
3. A student entry is also assumed to be present in the database. The credentials for this student is student13 as username and password as password.
4. The assignment created for testing contain no teams and topic selections.

Steps for manual submission

Before analyzing code, one must make themselves familiar with the steps involved in assignment submission by student on Expertiza. The steps involved are:

1. Login with valid student username and password
2. Click on an assignment to submit
3. Click on "Your Work"
4. Paste the link or browse to the file containing your work
5. Click on "Upload Link" or "Upload File" to submit deliverable

Creating the Tests

Gems involved

The gems used in feature testing are rspec-rails and capybara

rspec-rails

Rspec-rails is a testing framework for Rails 3.x and 4.x. It supports testing of models, controllers, requests, features, views and routes. It does this by accepting test scenarios called specs.<ref>rpsce-rails on GitHub</ref>

capybara

Capybara helps you test web applications by simulating how a real user would interact with your app. It is agnostic about the driver running your tests and comes with Rack::Test and Selenium support built in. WebKit is supported through an external gem.<ref>capybara on GitHub</ref>

Test Scenarios

Based on the steps involved in manual submission of the assignment, the following test scenarios are considered:

Scenario to check whether student is able to login Login with student13 and password Click on Sign In button Page should have text content Assignment

Scenario to check whether student is able to submit valid link to an ongoing assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''OnGoing Assignment''' Click_on 'Your work' Upload Link '''http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule''' Page should have text content '''http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule''''

Scenario to check whether student is not able submit invalid link to an ongoing assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''OnGoing Assignment''' Click_on 'Your work' Upload Link 'http://' Page should display flash message with text 'URI is not valid'

Scenario to check whether student is able to upload a file to an ongoing assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''OnGoing Assignment''' Click_on 'Your work' Upload file student_submission_spec.rb' Page should contain filename 'student_submission_spec.rb'

Scenario to check whether student is able to upload valid link and a file to an ongoing assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''OnGoing Assignment''' Click_on 'Your work' Upload Link 'http://www.csc.ncsu.edu/faculty/efg/517/f15/assignments' Upload file 'users_spec.rb' Page should have text content 'http://www.csc.ncsu.edu/faculty/efg/517/f15/assignments' Page should contain filename 'users_spec.rb'

Scenario to check whether student is not able submit valid link to a finished assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''Finished Assignment''' Click_on 'Your work' Page should not have 'Upload link' button

Scenario to check whether student is able upload file to a finished assignment Login with '''student13''' and '''password''' Click on '''Sign In''' button Click_on '''Finished Assignment''' Click_on 'Your work' Upload file 'student_submission_spec.rb' Page should contain filename 'student_submission_spec.rb'

Code

Code for the feature test consist in two files assignment_creation.rb and student_assignment_submission.rb.

assignment_creation.rb

As the name mentions, this file consist code for creating assignments. When this file is run, two assignments are created: one in ongoing/submission phase and other in finished state. The name of the ongoing assignment is FeatureTest and the name of the finished assignment is LibraryRailsApp. Furthermore this file also adds student13 as participant to both the assignments. The steps in this file are written in capybara using rspec framework. And the scenario is run with js enabled, so the tester can view the steps in a browser.

require 'rails_helper'
require 'spec_helper'

RSpec.feature 'assignment creation' do

  before(:each) do
    # Login as instructor
    visit root_path
    fill_in('login_name', :with => 'instructor6')
    fill_in('login_password', :with => 'password')
    click_on('SIGN IN')
    expect(page).to have_content('Manage')

    # Steps to create new ongoing public assignment

    # Step 1: Browse to new public assignment
    within('.content') do
      click_on('Assignments')
    end
    click_button 'New public assignment'

    # Step 2: Fill in details of new ongoing public assignment
    fill_in('assignment_form_assignment_name',:with => 'FeatureTest')
    select('CSC 517 Fall 2010', from: 'assignment_form_assignment_course_id')
    fill_in('assignment_form_assignment_directory_path',:with => 'csc517/oss')
    fill_in('assignment_form_assignment_spec_location',:with => 'feature test')
    check('assignment_form_assignment_availability_flag')
    click_on('Create')

    # Step 3: Specify grading rubrics for new ongoing public assignment
    click_on('Rubrics')
    within('#questionnaire_table_ReviewQuestionnaire') do
      select('Animation', from: 'assignment_form[assignment_questionnaire][][questionnaire_id]')
    end
    within('#questionnaire_table_AuthorFeedbackQuestionnaire') do
      select('Author feedback OTD1', from: 'assignment_form[assignment_questionnaire][][questionnaire_id]')
    end
    expect(page).to have_content("Rubrics")
    click_on('submit_btn')
    expect(page).to have_content("successfully",:wait=>5)

    # Step 4: Specify due dates for the new ongoing public assignment
    click_on('Due dates')
    fill_in('datetimepicker_submission_round_1',:with => '2015/11/10 23:00')
    fill_in('datetimepicker_review_round_1',:with => '2015/11/10 23:00')
    click_on('submit_btn')

    # Step 5: Add student13 as a participant to the new ongoing public assignment
    click_on('Manage...', :wait=>5)

    within(:xpath, "//table/tbody/tr[.//td[contains(.,'FeatureTest')]]") do
      find(:xpath, ".//a/img[@src='/assets/tree_view/add-participant-24.png']/..").click
    end
    expect(page).to have_content("Participants",:wait=>5)
    fill_in('user_name', :with => 'student13')
    click_on('add_a')

    # Steps to create new finished public assignment
    click_on('Manage...', :wait=>5)

    # Step 1: Browse to new public assignment
    click_button 'New public assignment'

    # Step 2: Fill in details of new finished public assignment
    fill_in('assignment_form_assignment_name',:with => 'LibraryRailsApp')
    select('CSC 517 Fall 2010', from: 'assignment_form_assignment_course_id')
    fill_in('assignment_form_assignment_directory_path',:with => 'csc517/rails')
    fill_in('assignment_form_assignment_spec_location',:with => 'rails application')
    check('assignment_form_assignment_availability_flag')
    click_on('Create')

    # Step 3: Specify grading rubrics for new finished public assignment
    click_on('Rubrics')
    within('#questionnaire_table_ReviewQuestionnaire') do
      select('Animation', from: 'assignment_form[assignment_questionnaire][][questionnaire_id]')
    end
    within('#questionnaire_table_AuthorFeedbackQuestionnaire') do
      select('Author feedback OTD1', from: 'assignment_form[assignment_questionnaire][][questionnaire_id]')
    end
    expect(page).to have_content("Rubrics")
    click_on('submit_btn')
    expect(page).to have_content("successfully",:wait=>5)

    # Step 4: Specify due dates for the new finished public assignment
    click_on('Due dates')
    fill_in('datetimepicker_submission_round_1',:with => '2015/9/9 23:00')
    fill_in('datetimepicker_review_round_1',:with => '2015/9/9 23:00')
    click_on('submit_btn')

    # Step 5: Add student13 as a participant to the new finished public assignment
    click_on('Manage...',:wait=>5)

    within(:xpath, "//table/tbody/tr[.//td[contains(.,'LibraryRailsApp')]]") do
      find(:xpath, ".//a/img[@src='/assets/tree_view/add-participant-24.png']/..").click
    end
    expect(page).to have_content("Participants",:wait=>5)
    fill_in('user_name', :with => 'student13')
    click_on('add_a')

  end


  scenario 'checked', :js => true do

    click_on('Manage...',:wait=>5)

    expect(page).to have_content 'FeatureTest'
    expect(page).to have_content 'LibraryRailsApp'

    # Logout
    find(:xpath, "//a[@href='/auth/logout']").click
  end

end

student_assignment_submission.rb

This file consists of the test scenarios discussed in the Test Scenario section. The steps mainly comprises of selecting either the FeatureTest or LibraryRailsApp assignment and submitting a link or file. Again these steps are written in capybara using RSpec. But unlike assignment_creation.rb, these tests run with js disabled and hence are executed without a browser. The following are the contents of the file:

require 'rails_helper'
require 'spec_helper'

RSpec.feature 'student assignment submission' do

  before(:each) do
    visit root_path
    fill_in 'User Name', :with => 'student13'
    fill_in 'Password', :with => 'password'
    click_on 'SIGN IN'
  end

  scenario 'student with valid credentials' do
    expect(page).to have_content 'Assignment'
  end

  scenario 'submitting only valid link to ongoing assignment' do
    click_on 'FeatureTest'
    click_on 'Your work'
    fill_in 'submission', :with => 'http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule'
    click_on 'Upload link'
    expect(page).to have_content 'http://www.csc.ncsu.edu/faculty/efg/517/f15/schedule'
  end

  scenario 'submitting only invalid link to ongoing assignment' do
    click_on 'FeatureTest'
    click_on 'Your work'
    fill_in 'submission', :with => 'http://'
    click_on 'Upload link'
    expect(page).to have_content 'bad URI(absolute but no path)'
  end

  scenario 'submitting only existing file to ongoing assignment' do
    click_on 'FeatureTest'
    click_on 'Your work'
    attach_file('uploaded_file', File.absolute_path('./spec/features/student_submission_spec.rb'))
    click_on 'Upload file'
    expect(page).to have_content 'student_submission_spec.rb'
  end

  scenario 'submitting link and file to ongoing assignment' do
    click_on 'FeatureTest'
    click_on 'Your work'
    fill_in 'submission', :with => 'http://www.csc.ncsu.edu/faculty/efg/517/f15/assignments'
    click_on 'Upload link'
    attach_file('uploaded_file', File.absolute_path('./spec/features/users_spec.rb'))
    click_on 'Upload file'
    expect(page).to have_content 'http://www.csc.ncsu.edu/faculty/efg/517/f15/assignments'
    expect(page).to have_content 'users_spec.rb'
  end

  scenario 'submitting link for finished assignment' do
    click_on 'LibraryRailsApp'
    click_on 'Your work'
    expect(page).to have_no_button('Upload link')
  end

  scenario 'submitting file for finished assignment' do
    click_on 'LibraryRailsApp'
    click_on 'Your work'
    expect(page).to have_no_button('Upload file')
  end

end

Running the tests

The following are steps required to run the test

• Clone the repository in a new directory

$ mkdir review
$ cd review
$ git clone https://github.com/shrenujgandhi/expertiza.git

In case you don't have a database with student and instructor entries then download the dump from https://drive.google.com/a/ncsu.edu/file/d/0B2vDvVjH76uESEkzSWpJRnhGbmc/view. Extract its contents. Open the terminal, traverse to that directory and type the following command

$ cd expertiza
$ rake db:create:all
$ mysql -u root -p expertiza_development < expertiza_scrubbed_2015_08_14.sql
password:

$ rake db:migrate

• Check if you are able to start rails server.

$ rails s

• Type the command to create assignment

rspec spec/features/assignment_creation.rb    

• Type the command to run tests for assignment submission

rspec spec/features/student_submission_spec.rb 

Test Results

The following is the result after running rspec spec/features/assignment_creation.rb

expertiza_developer@expertizadev: ~/review/expertiza master
$ rspec spec/features/assignment_creation.rb                                                               [1:39:18]
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.

Randomized with seed 34268
.

Finished in 5 minutes 31 seconds (files took 3.92 seconds to load)
1 example, 0 failures

Randomized with seed 34268

Coverage report generated for RSpec to /home/expertiza_developer/review/expertiza/coverage. 1688 / 5005 LOC (33.73%) covered.
[Coveralls] Outside the CI environment, not sending data.
 
expertiza_developer@expertizadev: ~/review/expertiza master
$        

0 failures mean that the assignments "FeatureTest" and "LibraryRailsApp" were created successfully.

The following is the result after running rspec spec/features/student_submission_spec.rb

expertiza_developer@expertizadev: ~/review/expertiza master
$ rspec spec/features/student_submission_spec.rb                                                           [1:45:15]
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.

Randomized with seed 53903
...F...

Failures:

  1) student assignment submission submitting file for finished assignment
     Failure/Error: expect(page).to have_no_button('Upload file')
       expected #has_no_button?("Upload file") to return true, got false
     # ./spec/features/student_submission_spec.rb:61:in `block (2 levels) in <top (required)>'

Finished in 29.37 seconds (files took 5.57 seconds to load)
7 examples, 1 failure

Failed examples:

rspec ./spec/features/student_submission_spec.rb:58 # student assignment submission submitting file for finished assignment

Randomized with seed 53903

Coverage report generated for RSpec to /home/expertiza_developer/review/expertiza/coverage. 1297 / 4307 LOC (30.11%) covered.
[Coveralls] Outside the CI environment, not sending data.
FAIL: 1

expertiza_developer@expertizadev: ~/review/expertiza master
$       

7 examples, 1 failure means that our scenarios were executed but one of them failed. This scenario is "submit file for finished assignment" which states that, unlike "upload link" button, the "Upload file" button is still active. This means that user can submit a file even after the assignment is finished.

Existing Bugs

During the course of our testing, we encountered two potential bugs in the Expertiza system. These are:

1. A student can upload file even after the assignment is finished.
2. A student can upload an .exe file to any assignment. Basically there is no check present on the type of file uploaded.

The following screenshots show these bugs on the current version of Expertiza used by the batch of Fall 2015.

Figure: This screenshot shows the list of assignments along with their current stage in Expertiza.

Figure: This screenshot shows that a student can select a file in a finished assignment in Expertiza.

Figure: This screenshot shows that a student can upload a file in a finished assignment in Expertiza.

Figure: This screenshot shows that a student can select an .exe file in any assignment in Expertiza.

Figure: This screenshot shows that a student can upload an .exe file in any assignment in Expertiza.

Project Resources

1. GitHub Link
2. YouTube Demo

References

<references></references>

External links

• Expertiza Documentation