CSC/ECE 517 Fall 2016/E1664: Feature Test Assignment Creation

Background

Expertiza is an open source web-based platform for the organizing and management of student project submissions and their evaluation, that supports augmentation of subject knowledge and accelerated learning via peer reviews. The application is meant for two types of users, instructors and students to use the application. Instructor can create and manage a course, enroll students in the course, create, manage and personalize assignments, project submissions and reviews for the particular course. Students can make their project submissions for the assignments allocated to them by their instructors for a course, give peer reviews, teammate reviews, create quizzes for their project reviewers, provide feedbacks and view their grades.

Objective

Assignment Creation can be done in Expertiza by an instructor for a particular course. Two types of assignments can be created, Private and Public. Further, there are various ways an assignment's attributes can be modified by personalizing reviews, deadlines, teams allocation, rubrics, calibration etc associated with the particular assignment. Each one of these in turn have many other options through which an instructor can be selected and modified. There are many functionalities within assignment creation, thus the main motive for testing of this feature is to ensure that an instructor can create an assignment without any hick-ups with all of its properties intact. And this is

Feature Testing Tools

Test cases have been written using RSpec and Capybara. These two combined prove to be an excellent testing tool at the developer's disposal.

Rspec

RSpec is a very powerful testing tool primarily used Behavior Driven Development (BDD). In RSpec, test cases are written such that they describe the specific functionality in the system being tested and helps user in better visualization of what the test cases are supposed to do.

Capybara

Capybara is another extremely helpful testing tool specifically used for feature testing. It is a web-based automation framework, and coupled with RSpec allows developer to write the test cases which simulate the whole scenario. It comes with a user friendly DSL as the actions specified are very intuitive and easy to understand and follow.

Test Cases

Newly Added Test Cases

1. Test Assignment Creation

a. Test Creating public Assignment

   it "is able to create a public assignment" do
     login_as("instructor6")
     visit "/assignments/new?private=0"
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_reviews_visible_to_all")
     check("assignment_form_assignment_is_calibrated")
     uncheck("assignment_form_assignment_availability_flag")
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       name: 'public assignment for test',
       course_id: Course.find_by_name('Course 2')[:id],
       directory_path: 'testDirectory',
       spec_location: 'testLocation',
       microtask: true,
       is_calibrated: true,
       availability_flag: false
     )
   end

b. Test Creating private Assignment

   it "is able to create a private assignment" do
     login_as("instructor6")
     visit "/assignments/new?private=1"
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_reviews_visible_to_all")
     check("assignment_form_assignment_is_calibrated")
     uncheck("assignment_form_assignment_availability_flag")
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       name: 'private assignment for test',
       course_id: Course.find_by_name('Course 2')[:id],
       directory_path: 'testDirectory',
       spec_location: 'testLocation'
       )
   end 

c. Create Assignment for Team

   it "is able to create with teams" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("team_assignment")
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 3
     click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       max_team_size: 3,
       show_teammate_reviews: true
       )
   end 

d. Create Assignment with Quiz

   it "is able to create with quiz" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("assignment_form_assignment_require_quiz")
     click_button 'Create'
     fill_in 'assignment_form_assignment_num_quiz_questions', with: 3
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
       num_quiz_questions: 3,
       require_quiz: true
     )
  end

e. Create Assignment with staggered deadline

   it "is able to create with staggered deadline" do
     skip('skip test on staggered deadline temporarily')
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     begin
       check("assignment_form_assignment_staggered_deadline")
     rescue
       return
     end
     page.driver.browser.switch_to.alert.accept
     click_button 'Create'
     fill_in 'assignment_form_assignment_days_between_submissions', with: 7
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'private assignment for test').first
     pending(%(not sure what's broken here but the error is: #ActionController::RoutingError: No route matches [GET] 
                 "/asset/staggered_deadline_assignment_grap/graph_1.jpg"))
     expect(assignment).to have_attributes(
       staggered_deadline: true
     )
   end
  

f. Create Assignment with review visible to all reviewers

   it "is able to create with review visible to all reviewers" do
     login_as("instructor6")
     visit '/assignments/new?private=1'
     fill_in 'assignment_form_assignment_name', with: 'private assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
     check('assignment_form_assignment_reviews_visible_to_all')
     click_button 'Create'
     expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])
     #click_button 'Create'
     assignment = Assignment.where(name: 'private assignment for test').first
     expect(assignment).to have_attributes(
                               name: 'private assignment for test',
                               course_id: Course.find_by_name('Course 2')[:id],
                               directory_path: 'testDirectory',
                               spec_location: 'testLocation')
   end

g. Create public micro-task assignment

  it "is able to create public micro-task assignment" do
     login_as("instructor6")
     visit '/assignments/new?private=0'
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check('assignment_form_assignment_microtask')
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(microtask: true)
  end

h. create calibrated public assignment

  it "is able to create calibrated public assignment" do
     login_as("instructor6")
     visit '/assignments/new?private=0'
     fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
     check("assignment_form_assignment_is_calibrated")
     click_button 'Create'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
                               is_calibrated: true)
  end

2. Test General Tab of Assignment Creation

a. Edit assignment available to students

  it "should edit assignment available to students" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_microtask")
     check("assignment_form_assignment_is_calibrated")
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         microtask: true,
         is_calibrated: true,
     )
  end

b. Edit number of quizzes available to students

  it "should edit quiz number available to students" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_require_quiz")
     click_button 'Save'
     fill_in 'assignment_form_assignment_num_quiz_questions', with: 5
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         num_quiz_questions: 5,
         require_quiz: true
     )
  end

c. Edit number of members per team in an Assignment

   it "should edit number of members per team " do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 5
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1',
         max_team_size: 5,
         show_teammate_reviews: true
       )
    end

e. Edit review visible to all other reviewers'

   it "should edit review visible to all other reviewers" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check ("assignment_form_assignment_reviews_visible_to_all")
     click_button 'Save'
     assignment = Assignment.where(name: 'edit assignment for test').first
     expect(assignment).to have_attributes(
         name: 'edit assignment for test',
         course_id: Course.find_by_name('Course 2')[:id],
         directory_path: 'testDirectory1',
         spec_location: 'testLocation1'
     )
   end

f. Should create teammate review row in rubrics

  it "should create teammate review row in rubrics" do
     fill_in 'assignment_form_assignment_name', with: 'edit assignment for test'
     select('Course 2', from: 'assignment_form_assignment_course_id')
     fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory1'
     fill_in 'assignment_form_assignment_spec_location', with: 'testLocation1'
     check("team_assignment")
     check("assignment_form_assignment_show_teammate_reviews")
     fill_in 'assignment_form_assignment_max_team_size', with: 5
     click_button 'Save'
     click_link 'Rubrics'
     expect page.should have_css("table#assignment_questionnaire_table tr", :count=>4)
  end

g. Check if checking calibration shows the tab

   it "check if checking calibration shows the tab" do
     uncheck 'assignment_form_assignment_is_calibrated'
     click_button 'Save'
     check 'assignment_form_assignment_is_calibrated'
     click_button 'Save'
     expect(page).to have_selector('#Calibration')
   end

3. Test Topic Tab of Assignment Creation

a. edit topics properties - Check all options

  it "can edit topics properties - Check" do
     check("assignment_form_assignment_allow_suggestions")
     check("assignment_form_assignment_is_intelligent")
     check("assignment_form_assignment_can_review_same_topic")
     check("assignment_form_assignment_can_choose_topic_to_review")
     check("assignment_form_assignment_use_bookmark")
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       allow_suggestions: true,
       is_intelligent: true,
       can_review_same_topic: true,
       can_choose_topic_to_review: true,
       use_bookmark: true
     )
   end
         

b. edit topics properties- uncheck all options

 it "can edit topics properties - unCheck" do
     uncheck("assignment_form_assignment_allow_suggestions")
     uncheck("assignment_form_assignment_is_intelligent")
     uncheck("assignment_form_assignment_can_review_same_topic")
     uncheck("assignment_form_assignment_can_choose_topic_to_review")
     uncheck("assignment_form_assignment_use_bookmark")
     click_button 'submit_btn'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       allow_suggestions: false,
       is_intelligent: false,
       can_review_same_topic: false,
       can_choose_topic_to_review: false,
       use_bookmark: false
     )
 end

c. Add new topic

  it "Add new topic" do
     click_link 'New topic'
     click_button 'OK'
     fill_in 'topic_topic_identifier', with: '1'
     fill_in 'topic_topic_name', with: 'Test'
     fill_in 'topic_category', with: 'Test Category'
     fill_in 'topic_max_choosers', with: 2
     click_button 'Create'
     sign_up_topics = SignUpTopic.where(topic_name: 'Test').first
     expect(sign_up_topics).to have_attributes(
       topic_name: 'Test',
       assignment_id: 1,
       max_choosers: 2,
       topic_identifier: '1',
       category: 'Test Category'
        )
   end

d. Delete existing topic

   it "Delete existing topic" do
     create(:sign_up_topic, assignment_id: @assignment[:id])
     visit "/assignments/#{@assignment[:id]}/edit"
     click_link 'Topics'
     all(:xpath, '//img[@title="Delete Topic"]')[0].click
     click_button 'OK'
     topics_exist = SignUpTopic.count(:all, assignment_id: @assignment[:id])
     expect(topics_exist).to be_eql 0
   end

4. Test Rubrics Tab of Assignment Creation

a. Update review questionnaire rubric

   it "updates review questionnaire" do
         within(:css, "tr#questionnaire_table_ReviewQuestionnaire") do
         select "ReviewQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         uncheck('dropdown')
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
         fill_in 'assignment_form[assignment_questionnaire][][questionnaire_weight]', with: '50'
         fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', with: '50'
       end
       click_button 'Save'
       sleep 1
       questionnaire = get_questionnaire("ReviewQuestionnaire2").first
       expect(questionnaire).to have_attributes(
         questionnaire_weight: 50,
         notification_limit: 50
       )
     end
    

b. Update scored question dropdown

  it "should update scored question dropdown" do
       within("tr#questionnaire_table_ReviewQuestionnaire") do
         select "ReviewQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
       end
       click_button 'Save'
       questionnaire = Questionnaire.where(name: "ReviewQuestionnaire2").first
       assignment_questionnaire = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).first
       expect(assignment_questionnaire.dropdown).to eq(false)
  end

c. Updates author feedback questionnaire

    it "updates author feedback questionnaire" do
       within(:css, "tr#questionnaire_table_AuthorFeedbackQuestionnaire") do
         select "AuthorFeedbackQuestionnaire2", from: 'assignment_form[assignment_questionnaire][][questionnaire_id]'
         uncheck('dropdown')
         select "Scale", from: 'assignment_form[assignment_questionnaire][][dropdown]'
         fill_in 'assignment_form[assignment_questionnaire][][questionnaire_weight]', with: '50'
         fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', with: '50'
       end
       click_button 'Save'
       questionnaire = get_questionnaire("AuthorFeedbackQuestionnaire2").first
       expect(questionnaire).to have_attributes(
         questionnaire_weight: 50,
         notification_limit: 50
       )
   end

5. Test Review Strategy of Assignment Creation

a. Test all auto selects feature

   it "auto selects" do
     login_as("instructor6")
     visit "/assignments/#{@assignment_id}/edit"
     find_link('ReviewStrategy').click
     select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
     fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
     fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
     click_button 'Save'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
       review_assignment_strategy: 'Auto-Selected',
       review_topic_threshold: 3,
       max_reviews_per_submission: 10
     )
   end

6. Test Due dates tab of assignment Creation

a. loads the due dates page

  it "it loads the due dates page" do
     expect(page).to have_content("Number of review rounds")
  end

b. set the deadline for an assignment review

  it "set the deadline for an assignment review" do
     fill_in 'assignment_form_assignment_rounds_of_reviews', with: '1'
     fill_in 'datetimepicker_submission_round_1', with: '2017/10/01 12:00'
     fill_in 'datetimepicker_review_round_1', with: '2017/10/10 12:00'
     click_button 'submit_btn'
     submission_type_id = DeadlineType.where(name: 'submission')[0].id
     review_type_id = DeadlineType.where(name: 'review')[0].id
     submission_due_date = DueDate.find(1)
     review_due_date = DueDate.find(2)
     expect(submission_due_date).to have_attributes(
       deadline_type_id: submission_type_id,
       type: 'AssignmentDueDate'
     )
     expect(review_due_date).to have_attributes(
       deadline_type_id: review_type_id,
       type: 'AssignmentDueDate'
     )
  end  

7. Test Adding participants by Instructor

a. Add New Participants to assignment

   it "check to see if participants can be added" do
     student = create(:student)
     login_as('instructor6')
     assignment_id = Assignment.where(name: 'participants Assignment')[0].id
     visit "/participants/list?id=#{assignment_id}&model=Assignment"
     fill_in 'user_name', with: student.name
     choose 'user_role_participant'
     expect {
       click_button 'Add'
     }.to change {Participant.count}.by 1
   end

b. Verify assignments assigned to participants

  it "should display newly created assignment" do
     participant = create(:participant)
     login_as(participant.name)
     expect(page).to have_content("participants Assignment")
  end

8. Check if assignment can be added to a course

it "check to find if the assignment can be added to a course", js: true  do
   create(:assignment, course: nil, name: 'Test Assignment')
   create(:course, name: 'Test Course')
   course_id = Course.where(name: 'test Course')[0].id
   assignment_id = Assignment.where(name: 'Test Assignment')[0].id
   login_as('instructor6')
   visit "/assignments/associate_assignment_with_course?id=#{assignment_id}"
   choose "course_id_#{course_id}"
   click_button 'Save'
   assignment_row = Assignment.where(name: 'Test Assignment')[0]
   expect(assignment_row).to have_attributes(
     course_id: course_id
   )
 end

References

Github Expertiza[1]

RSpec [2]

Capybara [3]