CSC/ECE 517 Fall 2016/E1664: Feature Test Assignment Creation

Background

Expertiza is an open source web-based platform for the organizing and management of student project submissions and their evaluation, that supports augmentation of subject knowledge and accelerated learning via peer reviews. The application is meant for two types of users, instructors and students to use the application. Instructor can create and manage a course, enroll students in the course, create, manage and personalize assignments, project submissions and reviews for the particular course. Students can make their project submissions for the assignments allocated to them by their instructors for a course, give peer reviews, teammate reviews, create quizzes for their project reviewers, provide feedbacks and view their grades.

Objective

Assignment Creation can be done in Expertiza by an instructor for a particular course. Two types of assignments can be created, Private and Public. Further, there are various ways an assignment's attributes can be modified by personalizing reviews, deadlines, teams allocation, rubrics, calibration etc associated with the particular assignment. Each one of these in turn have many other options through which an instructor can be selected and modified. There are many functionalities within assignment creation, thus the main motive for testing of this feature is to ensure that an instructor can create an assignment without any hick-ups with all of its properties intact. And this is

Files Modified

• assignment_creation_spec.rb (Capybara spec file)
• factories.rb (FactoryGirl's factory file

Feature Testing Tools

Test cases have been written using RSpec and Capybara. These two combined prove to be an excellent testing tool at the developer's disposal.

Rspec

RSpec is a very powerful testing tool primarily used Behavior Driven Development (BDD). In RSpec, test cases are written such that they describe the specific functionality in the system being tested and helps user in better visualization of what the test cases are supposed to do.

Capybara

Capybara is another extremely helpful testing tool specifically used for feature testing. It is a web-based automation framework, and coupled with RSpec allows developer to write the test cases which simulate the whole scenario. It comes with a user friendly DSL as the actions specified are very intuitive and easy to understand and follow.

Test Cases

1. Fixes in existing broken Test Cases

Due to continual development of Expertiza, some of the existing test cases that were implemented with relatively older version of Expertiza were found broken at the starting of the project. To make the assignment creation testing complete, and to avoid covering the same scenarios again by adding new working test cases unnecessarily, these test case were fixed. Some of the fixes that were provided as part of this project are as follows:

1. Some of the functionalities for which the test cases were written have been remodeled so they had to be skipped.

2. Running the test cases were giving an error in the code, when fetching a row from the deadline type table.

• We noticed that the structure of the test file was wrong. The code to populate the the "deadline types" did not run, but were supposed to run for each test case. We made this code run for all test cases.

3. Rubrics test cases were searching for links that weren't present on the page. These issues were also fixed.

2. Modified and Newly Added Test Cases

1. Test Assignment Creation

a. Test Creating public Assignment: This test will check if public assignments are getting created properly or not with proper attributes.

   it "is able to create a public assignment" do
     login_as("instructor6")
     visit "/assignments/new?private=0"
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_reviews_visible_to_all")
     check("assignment_form_assignment_is_calibrated")
     uncheck("assignment_form_assignment_availability_flag")
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       name: 'public assignment for test',
       course_id: Course.find_by_name('Course 2')[:id],
       directory_path: 'testDirectory',
       spec_location: 'testLocation',
       microtask: true,
       is_calibrated: true,
       availability_flag: false
     )
   end

b. Test Creating private Assignment: This test will check if private assignments are getting created or not with proper attributes.

   it "is able to create a private assignment" do
     login_as("instructor6")
     visit "/assignments/new?private=1"
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_reviews_visible_to_all")
     check("assignment_form_assignment_is_calibrated")
     uncheck("assignment_form_assignment_availability_flag")
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       name: 'private assignment for test',
       course_id: Course.find_by_name('Course 2')[:id],
       directory_path: 'testDirectory',
       spec_location: 'testLocation'
       )
   end 

c. Create Assignment for Team: This test will check the creation of new private assignment for the team. The maximum size of the team is 3 and the test will check if all these features are there or not.

   it "is able to create with teams" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("team_assignment")
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 3
     click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       max_team_size: 3,
       show_teammate_reviews: true
       )
   end 

d. Create Assignment with Quiz: This test will check the creation of new private assignment with the quiz. The maximum number of questions in the quiz is 3 and the test will check if all these features are present or not.

   it "is able to create with quiz" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("assignment_form_assignment_require_quiz")
     click_button 'Create'
     fill_in 'assignment_form_assignment_num_quiz_questions', with: 3
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       num_quiz_questions: 3,
       require_quiz: true
     )
  end
  

e. Create Assignment with review visible to all reviewers: This test will check the creation of new private assignment with review visible to all the reviewers.

   it "is able to create with review visible to all reviewers" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check('assignment_form_assignment_reviews_visible_to_all')
     click_button 'Create'
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     #click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
                               name: 'private assignment for test',
                               course_id: Course.find_by_name('Course 2')[:id],
                               directory_path: 'testDirectory',
                               spec_location: 'testLocation')
   end

f. Create public micro-task assignment: This test will check creation of new public assignment with microtasks.

  it "is able to create public micro-task assignment" do
     login_as("instructor6")
     visit '/assignments/new?private=0'
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check('assignment_form_assignment_microtask')
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(microtask: true)
  end

g. Create calibrated public assignment: This test will check creation of new calibrated public assignment.

  it "is able to create calibrated public assignment" do
     login_as("instructor6")
     visit '/assignments/new?private=0'
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("assignment_form_assignment_is_calibrated")
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
                               is_calibrated: true)
  end

2. Test General Tab of Assignment Creation

a. Edit assignment available to students: This test will edit the existing assignment and enable micro tasks and calibrated options. The test will verify if all these options are updated in the assignment.

  it "should edit assignment available to students" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_is_calibrated")
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         microtask: true,
         is_calibrated: true,
     )
  end

b. Edit number of quizzes available to students: This test will check the feature of editing the number of quizzes in the existing assignment.

  it "should edit quiz number available to students" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_require_quiz")
     click_button 'Save'
     fill_in 'assignment_form_assignment_num_quiz_questions', with: 5
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         num_quiz_questions: 5,
         require_quiz: true
     )
  end

c. Edit number of members per team in an Assignment: This test will check the feature of editing the maximum number of members in a team for an assignment.

   it "should edit number of members per team " do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 5
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         max_team_size: 5,
         show_teammate_reviews: true
       )
    end

e. Edit review visible to all other reviewers: This test will verify the feature of adding/removing review visible to all the reviewers for an assignment.

   it "should edit review visible to all other reviewers" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check ("assignment_form_assignment_reviews_visible_to_all")
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1'
     )
   end

f. Should create teammate review row in rubrics: This test will verify that adding teams and team members in an assignment will create one more row of teammate review in rubrics.

  it "should create teammate review row in rubrics" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("team_assignment")
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 5
     click_button 'Save'
     click_link 'Rubrics'
     expect page.should have_css("table#assignment_questionnaire_table tr", :count=>4)
  end

g. Check if checking calibration shows the tab:

   it "check if checking calibration shows the tab" do
     uncheck 'assignment_form_assignment_is_calibrated'
     click_button 'Save'
     check 'assignment_form_assignment_is_calibrated'
     click_button 'Save'
     expect(page).to have_selector('#Calibration')
   end

3. Test Topic Tab of Assignment Creation

a. Edit topics properties - Check all options

  it "can edit topics properties - Check" do
     check("assignment_form_assignment_allow_suggestions")
     check("assignment_form_assignment_is_intelligent")
     check("assignment_form_assignment_can_review_same_topic")
     check("assignment_form_assignment_can_choose_topic_to_review")
     check("assignment_form_assignment_use_bookmark")
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       allow_suggestions: true,
       is_intelligent: true,
       can_review_same_topic: true,
       can_choose_topic_to_review: true,
       use_bookmark: true
     )
   end
         

b. Edit topics properties- uncheck all options

 it "can edit topics properties - unCheck" do
     uncheck("assignment_form_assignment_allow_suggestions")
     uncheck("assignment_form_assignment_is_intelligent")
     uncheck("assignment_form_assignment_can_review_same_topic")
     uncheck("assignment_form_assignment_can_choose_topic_to_review")
     uncheck("assignment_form_assignment_use_bookmark")
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       allow_suggestions: false,
       is_intelligent: false,
       can_review_same_topic: false,
       can_choose_topic_to_review: false,
       use_bookmark: false
     )
 end

c. Add new topic : This test will check feature of adding new topic for an assignment.

  it "Add new topic" do
     click_link 'New topic'
     click_button 'OK'
     fill_in 'topic_topic_identifier', with: '1'
     fill_in 'topic_topic_name', with: 'Test'
     fill_in 'topic_category', with: 'Test Category'
     fill_in 'topic_max_choosers', with: 2
     click_button 'Create'
     sign_up_topics = SignUpTopic.where(topic_name: 'Test').first
     expect(sign_up_topics).to have_attributes(
       topic_name: 'Test',
       assignment_id: 1,
       max_choosers: 2,
       topic_identifier: '1',
       category: 'Test Category'
        )
   end

d. Delete existing topic: This test will check feature of deleting new topic for an assignment.

   it "Delete existing topic" do
     create(:sign_up_topic, assignment_id: @assignment[:id])
     visit "/assignments/#{@assignment[:id]}/edit"
     click_link 'Topics'
     all(:xpath, '//img[@title="Delete Topic"]')[0].click
     click_button 'OK'
     topics_exist = SignUpTopic.count(:all, assignment_id: @assignment[:id])
     expect(topics_exist).to be_eql 0
   end

4. Test Rubrics Tab of Assignment Creation

a. Update review questionnaire rubric:

   it "updates review questionnaire" do
         within(:css, "tr#questionnaire_table_ReviewQuestionnaire") do
         select "ReviewQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         uncheck('dropdown')
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
         fill_in 'assignment_form[assignment_questionnaire][][questionnaire_weight]', with: '50'
         fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', with: '50'
       end
       click_button 'Save'
       sleep 1
       questionnaire = get_questionnaire("ReviewQuestionnaire2").first
       expect(questionnaire).to have_attributes(
         questionnaire_weight: 50,
         notification_limit: 50
       )
     end
    

b. Update scored question dropdown

  it "should update scored question dropdown" do
       within("tr#questionnaire_table_ReviewQuestionnaire") do
         select "ReviewQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
       end
       click_button 'Save'
       questionnaire = Questionnaire.where(name: "ReviewQuestionnaire2").first
       assignment_questionnaire = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).first
       expect(assignment_questionnaire.dropdown).to eq(false)
  end

c. Updates author feedback questionnaire

    it "updates author feedback questionnaire" do
       within(:css, "tr#questionnaire_table_AuthorFeedbackQuestionnaire") do
         select "AuthorFeedbackQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         uncheck('dropdown')
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
         fill_in 'assignment_form[assignment_questionnaire][][questionnaire_weight]', with: '50'
         fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', with: '50'
       end
       click_button 'Save'
       questionnaire = get_questionnaire("AuthorFeedbackQuestionnaire2").first
       expect(questionnaire).to have_attributes(
         questionnaire_weight: 50,
         notification_limit: 50
       )
   end

5. Test Review Strategy of Assignment Creation

a. Test all auto selects feature

   it "auto selects" do
     login_as("instructor6")
     visit "/assignments/#{@assignment_id}/edit"
     find_link('ReviewStrategy').click
     select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
     fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
     fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
     click_button 'Save'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       review_assignment_strategy: 'Auto-Selected',
       review_topic_threshold: 3,
       max_reviews_per_submission: 10
     )
   end

6. Test Due dates tab of assignment Creation

a. Loads the due dates page

  it "it loads the due dates page" do
     expect(page).to have_content("Number of review rounds")
  end

b. Set the deadline for an assignment review: This function will test if deadlines are getting assigned properly or not for the review.

  it "set the deadline for an assignment review" do
     fill_in 'assignment_form_assignment_rounds_of_reviews', with: '1'
     fill_in 'datetimepicker_submission_round_1', with: '2017/10/01 12:00'
     fill_in 'datetimepicker_review_round_1', with: '2017/10/10 12:00'
     click_button 'submit_btn'
     submission_type_id = DeadlineType.where(name: 'submission')[0].id
     review_type_id = DeadlineType.where(name: 'review')[0].id
     submission_due_date = DueDate.find(1)
     review_due_date = DueDate.find(2)
     expect(submission_due_date).to have_attributes(
       deadline_type_id: submission_type_id,
       type: 'AssignmentDueDate'
     )
     expect(review_due_date).to have_attributes(
       deadline_type_id: review_type_id,
       type: 'AssignmentDueDate'
     )
  end  

7. Test Adding participants by Instructor

a. Add New Participants to assignment: This test will verify if an instructor is able to assign new participants to an assignment.

   it "check to see if participants can be added" do
     student = create(:student)
     login_as('instructor6')
     assignment_id = Assignment.where(name: 'participants Assignment')[0].id
     visit "/participants/list?id=#{assignment_id}&model=Assignment"
     fill_in 'user_name', with: student.name
     choose 'user_role_participant'
     expect {
       click_button 'Add'
     }.to change {Participant.count}.by 1
   end

b. Verify assignments assigned to participants: This test will verify if assignments are properly assigned to participants. Participants should be able to see newly assigned assignment in their page.

  it "should display newly created assignment" do
     participant = create(:participant)
     login_as(participant.name)
     expect(page).to have_content("participants Assignment")
  end

8. Check if assignment can be added to a course: It will check an assignment can be added to the course

it "check to find if the assignment can be added to a course", js: true  do
   create(:assignment, course: nil, name: 'Test Assignment')
   create(:course, name: 'Test Course')
   course_id = Course.where(name: 'test Course')[0].id
   assignment_id = Assignment.where(name: 'Test Assignment')[0].id
   login_as('instructor6')
   visit "/assignments/associate_assignment_with_course?id=#{assignment_id}"
   choose "course_id_#{course_id}"
   click_button 'Save'
   assignment_row = Assignment.where(name: 'Test Assignment')[0]
   expect(assignment_row).to have_attributes(
     course_id: course_id
   )
 end

Test the new code

We have uploaded video showing how the tests are run and their result. If user wants to test the newly added feature test, please clone the master branch of our git repository and perform following steps:

1. git clone http://github.com/dbzvishal/expertiza/

2. cd expertiza

3. rspec spec/features/assignment_creation_spec.rb

Automated Execution via Travis

Activated github repository in travis. Whenever we push changes in our repository, travis will start building and run all test cases. At the end, it will give the final result consisting of the number of test cases passed and failed and the overall build result. Travis result [1]

References

1. Github Expertiza[2]

2. RSpec [3]

3. Capybara [4]