Talk:CSC/ECE 517 Fall 2016
Review requirements and thresholds
Background
In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0). That’s wrong; I should always be allowed to review the work with the least reviews that I have not already reviewed. And that should generalize to situations in which k > 0. Another issue is that there is no way for Expertiza to tell a reviewer how many reviews are required. In the case of instructor-selected reviewing, that’s not a problem, but for auto-selected reviewing, there is no way to specify how many reviews are required (or even how many are allowed, in case students are allowed to do extra reviews).
Purpose
The purpose of this task is to solve the above mentioned issues in collaborative ways. The assignments table now contains the Max reviews required and allowed field which can be set from Review strategy tab. This allows the admin to set the threshold value for required number of reviews for the assignment. Another feature implemented is, now students can check how many reviews he/she needs to write which helps students determine how many reviews to write and also how many reviews he/she has already written.
External Links
- link for forked repository