CSC/CSC 517 Spring 2017/oss E1718

E1718 was the project code for an Expertiza OSS assignment. It largely dealt with the handling of how peer reviews are assigned to students.

About Expertiza

The Expertiza project is a collaborative open source project that allows courses to create assignments for students. One of the main functions of Expertiza is to allow students to peer review the submitted assignments by their fellow classmates.

E1718

This page covers the E1718 project. E1718 covers the system behind assigning peer reviews to students. The main focus of this topic is the thresholds that guide which submissions a student can review as well as how many they need to review, and can review. A review can either be assigned to a student by the instructor, or it can be requested directly by the student. When the student handles it they can choose to be assigned a random submission, or they can choose the topic which they wish to review. Assignments can be set up so that each student must review a minimum number of submissions, with the choice of extra credit for completing up to a maximum number of reviews. When review topics are chosen, Expertiza sets it up so that the student can only choose a topic that has a low enough number of other reviews. At times this can constrain the choices a student has, so to fix this the instructor can set the threshold to allow for more choices. The threshold itself is the acceptable range for the number of reviews a topic has compared to the least reviewed topic. For instance, lets say there are three topics: Blue, Red and Green. Blue has had 1 review, Red has had 2 reviews, and Green has had 3 reviews. If the threshold is set to 0, then the only topic that can be chosen for review is Blue. If the threshold is 1, then both Blue and Red can be chosen. If it is 3, then all 3 topics can be chosen.

By solving the issues raised by E1718 this system will be improved in several ways. As currently set up, if a student has already reviewed all assignments within the minimum threshold, they will be unable to request another review. To solve this, when a student has reviewed all submissions within threshold k, the system should increase the threshold by m so that the student can review all submissions within threshold k+m. The current version of Expertiza also fails to show a student what the minimum number of reviews they must complete is, as well as the upper limit of reviews allowed. This will be fixed in this project so that an instructor can set the minimum and maximum number of reviews from the Review Strategy tab, and those limits show up on the student's end when viewing Others' Work. A previous issue that has since been solved was that a student was allowed to review their own submission. While this did not need further work, it did need automated testing to be written for it. Those tests were completed, as well as tests for the issues described above.

Issues

228: Increase the threshold for a student who has already completed reviews for all submissions with the minimum number.

402: Write automated tests to check if a student can review a topic only they have submitted on.

417: Implement a num_reviews_required (and num_reviews_allowed) field in the assignments table to say how many reviews per reviewer are required, and how many are allowed (default should be # allowed = # req’d.). Make it settable from the Review Strategy tab (see Issue 417) and viewable when a student clicks on “Others’ work”.