CSC/ECE 517 Fall 2016/E1660. Review requirements and thresholds
E1660. Review requirements and thresholds
Background
In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0). That’s wrong; I should always be allowed to review the work with the least reviews that I have not already reviewed. And that should generalize to situations in which k > 0. Another issue is that there is no way for Expertiza to tell a reviewer how many reviews are required. In the case of instructor-selected reviewing, that’s not a problem, but for auto-selected reviewing, there is no way to specify how many reviews are required (or even how many are allowed, in case students are allowed to do extra reviews).
Purpose
The purpose of this task is to solve the above mentioned issues collaboratively. The assignments table now contains the Max reviews required and allowed field which can be set from Review strategy tab. This allows the admin to set the threshold value for required number of reviews for the assignment. Another feature implemented is, now students can check how many reviews he/she needs to write which helps students determine how many reviews to write and also how many reviews he/she has already written. Another issue resolved was the reviewer who finished reviewing all submissions with least number of reviews, should be able to review the next set of submissions with greater number of reviews. That is, the threshold rule was more relaxed. Implemented the necessary test cases for the new features implemented.
Modified File
- app/views/assignments/edit/_review_strategy.html.erb
- db/schema.rb
- Added two new unit tests in RSpec file in spec/models/assignment_form_spec.rb
- Added two new columns in Assignment table as shown below
- app/models/review_assignment.rb
- app/controllers/review_mapping_controller.rb
- added new file review_assignment_spec.rb in spec for testing the above functionality.
ALTER TABLE assignments ADD COLUMN num_reviews_required INT DEFAULT 3; ALTER TABLE assignments ADD COLUMN num_reviews_allowed INT DEFAULT 3;
- add new RSpec tests in assignment_spec.rb
describe "can not review own work", type: :feature do
before(:each) do
create(:assignment, name: "TestTeam", directory_path: 'test_team')
create_list(:participant, 3)
create(:assignment_node)
create(:deadline_type, name: "submission")
create(:deadline_type, name: "review")
create(:deadline_type, name: "metareview")
create(:deadline_type, name: "drop_topic")
create(:deadline_type, name: "signup")
create(:deadline_type, name: "team_formation")
create(:deadline_right)
create(:deadline_right, name: 'Late')
create(:deadline_right, name: 'OK')
create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now + (100 * 24 * 60 * 60))
#create(:topic)
create(:topic, topic_name: "work1")
end
it "case1" do
#test log in as a student
user = User.find_by_name('student2064')
msg = user.to_yaml
File.open('log/diagnostic.txt', 'a') {|f| f.write msg }
visit root_path
fill_in 'login_name', with: 'student2064'
fill_in 'login_password', with: 'password'
click_button 'SIGN IN'
#expect(page).to have_content "Welcome!"
#expect(page).to have_content "Assignments"
#click_link "Assignments"
expect(page).to have_content "User: student2064"
expect(page).to have_content "TestTeam"
click_link "TestTeam"
expect(page).to have_content "Signup sheet"
expect(page).to have_content "Your team"
#test if the topic can be seen and chosen by a student
click_link "Signup sheet"
expect(page).to have_content "work1"
my_link = find(:xpath, "//a[contains(@href,'sign_up_sheet/sign_up?assignment_id=#{Assignment.last.id}&id=1')]")
my_link.click
#test after selecting a topic, a team formed
click_link "Assignments"
click_link "TestTeam"
click_link "Your work"
fill_in 'submission', with: 'www.google.com'
click_button 'Upload link'
click_link "Assignments"
click_link "TestTeam"
click_link "Others' work"
click_button "Request a new submission to review"
#If no link of view is on the page, then it means that no review is allowed for this user.
expect(page).to have_no_link "view"
#save_and_open_page
end
end
RSpec testing
The new RSpec test added to the assignment_spec.rb is used to verify that the reviewer who submits their assignment should not be allowed to review their own work. The test mainly simulates mouse click on the webpage and tries to find certain content. One student account is created to submit the assignment which is stored by the system. Since this submission is the only one for the existed topic, the system is supposed to not to assign reviewer anything, which is equivalent to have the correct output for code, expect(page).to have_no_link "view". The test file shows desired result and the issue resolved.