CSC/ECE 517 Spring 2024/ OSS E2400 Allow Reviewers to Bid on What to Review

This wiki page provides information regarding the refactoring of E2400 review_mapping_controller.rb OSS project for Spring 2024, CSC/ECE 517

Introduction

The Expertiza project makes use of peer review to allow students to learn from one another. It's a Ruby on Rails-based open-source application. It is used by faculty and students for the management of courses and assignments for specific courses. Different screens in the manage content area of the application offer information about users, courses, assignments, questionnaires, and reviews.

A review bid is a system used in Expertiza, where students indicate their preferences for reviewing certain assignments or projects. Students bid on the tasks they prefer to review through ranking their interests. The allocation of review assignments is determined by a bidding algorithm, balancing student preferences with fair and efficient distribution.

Test Login Credentials

UserId: instructor6

Password: password

Problem Statement

1. The process of reviewing plays a crucial role in the learning and improvement cycle, benefiting both the reviewer and the reviewee. Allowing reviewers to choose their topics freely can lead to more thorough reviews and increased engagement. However, the current system of assigning review topics on a first-come-first-served basis disadvantages students who are slower to sign up, preventing them from reviewing topics they are most interested in.
2. An improved approach would be to introduce a bidding system for review topics, similar to the one used for topic selection. This system would enable students to express their preferences for topics they are keen to review. An effective bidding algorithm could ensure that all interested students receive a topic they are eager to review.
3. While the prior work met the functional requirements, it could not be integrated due to numerous violations of the DRY (Don't Repeat Yourself) principle.

Background

When topics are opened up for bidding, students can see how “hot” each topic is by the color it has on their topic list. However, instructors have no way to view the bidding process except by impersonating students. Furthermore, when the bidding assignment algorithm is run, there is no way to verify that it did in fact assign teams to topics they had chosen.

Feature Overview

Current Topic Bidding Feature

Students may bid on topics they would like to work on. The following image shows the functionality of this feature.

Students are allowed to bid on several topics, ranking them by priority. When the bidding deadline passes, every student is allocated a topic based on their preferences.

This allocation process utilizes a modified Gale-Shapely Algorithm to ensure a stable match between students and topics, aiming to satisfy student preferences as much as possible. To resolve preference ties, topics prioritize students based on whether they are bidding as part of a group (with group members receiving higher priority) and the timing of their bid (earlier bids receive higher priority). After the bidding deadline:

1. Students are initially paired with their top-choice topic.
2. If a topic exceeds its capacity, it retains only the students with the highest priority.
3. Topics that have reached capacity are then removed from the selection pool.
4. Unmatched students repeat the process, selecting from the remaining topics.
5. This cycle continues until every student has been assigned a topic.
6. The algorithm checks for and corrects any 'unstable' matches, where swapping students between topics would more accurately fulfill their preferences.

This process, with a time complexity of O(n^2), ensures that all students are matched with a topic they prefer, respecting their choices as closely as possible. For further information on this algorithm and its application, one can refer to documentation from the previous year when this method was first implemented.

Preferred Features for Reviews

The following Use Case Diagram demonstrates the primary functionality explained in the previous section for different types of users.

The desired feature for reviews should have a nearly same layout as this, but with reviews rather than themes. Students should be presented a list of topics that have been worked on, and they should be able to bid on the ones they want to review the most. The Priority system might be implemented in the same way, with the exception that a student should not be able to review their own work.

This feature appears as an option when editing/creating a new assignment under the "Topics" area, as shown in the interface drop down below.