CSC/ECE 517 Spring 2014/oss E1402 mmb: Difference between revisions
| Line 3: | Line 3: | ||
There are three different review strategies in Expertiza. | There are three different review strategies in Expertiza. | ||
<br>Auto Selected | <br>Auto Selected | ||
Student Selected | <br>Student Selected | ||
Instructor Selected | <br>Instructor Selected | ||
In auto-selected reviewing, a reviewer picks the topic to review on, but not the individual submission. | <br>In auto-selected reviewing, a reviewer picks the topic to review on, but not the individual submission. | ||
Student-selected allows the reviewer to pick from a list of submissions, not topics. | Student-selected allows the reviewer to pick from a list of submissions, not topics. | ||
Instructor-selected is where the instructor/TA assigns submissions to reviewers. | Instructor-selected is where the instructor/TA assigns submissions to reviewers. | ||
In the case of auto-selected and student-selected reviewing, the student is not allowed to pick from any topic or submission, but only to choose among those that have received the fewest reviews so far. But this strategy may be modified by selecting a non-zero threshold k as follows. | In the case of auto-selected and student-selected reviewing, the student is not allowed to pick from any topic or submission, but only to choose among those that have received the fewest reviews so far. But this strategy may be modified by selecting a non-zero threshold k as follows. | ||
A topic is review able if the minimum number of reviews already done for the submissions on that topic is within k of the minimum number of reviews done on the least-reviewed submission on any topic. | A topic is review able if the minimum number of reviews already done for the submissions on that topic is within k of the minimum number of reviews done on the least-reviewed submission on any topic. | ||
Revision as of 19:34, 31 March 2014
Background
There are three different review strategies in Expertiza.
Auto Selected
Student Selected
Instructor Selected
In auto-selected reviewing, a reviewer picks the topic to review on, but not the individual submission.
Student-selected allows the reviewer to pick from a list of submissions, not topics.
Instructor-selected is where the instructor/TA assigns submissions to reviewers.
In the case of auto-selected and student-selected reviewing, the student is not allowed to pick from any topic or submission, but only to choose among those that have received the fewest reviews so far. But this strategy may be modified by selecting a non-zero threshold k as follows.
A topic is review able if the minimum number of reviews already done for the submissions on that topic is within k of the minimum number of reviews done on the least-reviewed submission on any topic.