CSC/ECE 517 Fall 2021 - E2151. Allow reviewers to bid on what to review

Topic Overview

Statement of Problem

Assigning reviews to users is a complicated process. Currently, reviews are still assigned using a first-come-first-served basis. However, reviews could be bid on like how topics are bid on. This would involve matching multiple students to review a submission up to the maximum reviewers for a submission. A similar feature is present in bidding for topics.

The goal of this topic is to allow both reviews and topics to be bid on by students following a bidding algorithm, and for both of these features to come from the same root functionality. This will involve refactoring the current bidding functionality of expertiza, as well as incorporating new code to allow this functionality to also be utilized for reviews.

Prior Work

Before starting, it is vital to note the substantial amount of prior work done on this task. The prior work done in Fall of 2020 (linked here [[1]]) achieved much of the desired functionality for this task, and contains a comprehensive explanation of the underlying design principles behind the algorithm implemented. However, the implementation was rejected due to DRY violations. Thus, the primary goal of this project topic will be to rebuild and refactor this prior work to make it consistent with DRY principles and able to be merged into the expertiza environment.

A brief review will be given of the desired functionality and the underlying design for the bidding algorithm. For a more exhaustive explanation, please see the aforementioned link to the prior year's work.

Feature

In the current state of implementation the reviews are still assigned using a first-come-first-served basis. Current state of the feature implementation contains bad code smell and needs improvement to resolve merge conflict and follow DRY principle in order to finalize the implementation. With the current state in mind, the new implementation will enable the capability that reviews could be bid on like how topics are bid on. Behind the scene, the process would involve matching multiple students to review a submission up to the maximum reviewers for a submission.

Use Case Diagram

Edge Case Consideration

For the review workflow that consists of multiple review, each of the review could be bidded. Since we have had considered the case based on initial feedback, our current solution for such case is that the user shall indicate the number of reviews that a user want to review and algorithm will take the inputs into account. As for our implementation, we will focus more on the integration of the review bidding feature with the current system.

Explanation of Feature

Existing Bidding Functionality for Topics

Currently in expertiza, students may bid on which topics they would like to work on. An image showcasing this functionality is given below:

Students may choose a number of topics to bid on, in order of priority, and once the deadline is reached, all students are given a topic.

Behind the scenes, this works by utilizing a modified version of the Gale-Shapely Algorithm. This algorithm seeks to find a 'stable' matching of each student to each topic, such that in the end, all students have a topic they wanted (at least to some degree), and that the preferences of students is respected as well as possible. To help break situations where there are ties in preferences, each topic assigns a Priority to each student. This Priority is determined by A. whether the student is part of a group bidding for this topic (groups getting higher priority), and B. the time at which the bid for the topic was made (earlier bidders are given Priority). Once this has been determined and the deadline for bidding has ended, the following steps occur:

1. Each student is matched with each topic they most prefer.
2. If the number of students assigned to a topic is above the maximum allowed for that topic, the students with the highest Priority are assigned the topic
3. All maximally filled topics are removed from the pool of available topics
4. All unmatched students begin this cycle again, now choosing the most preferred topic remaining
5. Repeat until all students are assigned a topic
6. If any matches are 'unstable' (two students would have their preferences better met if they were swapped), perform any swaps until matches are as stable as possible.

This algorithm has a time complexity of O(n^2). For more details on this algorithm and its implementation, refer to the previous year's work, where this algorithm was implemented.

Desired Functionality for Reviews

The desired feature for reviews should have an almost identical structure to this, with reviews instead of topics. Students should be shown a list of the topics that were worked on, and may bid on which of those topics they would like to review the most. The Priority system could be handled in the same way, with the only caveat being that a student should not be able to review their own work.

Because of this similarity in functionality, our goal is to make both reviews and topics use the same underlying feature mechanisms.

Planned Changes

General Design Goals

• Update the prior work for this topic to be in line with the current state of the expertiza beta branch.
• DRY out prior implementation's controller methods by creating a single controller which handles bidding for both topics and reviews.
• DRY out prior implementation's views by implementing a view from scratch (rather than a code copy of topic bidding view).
• Create exhaustive tests for newly DRY-ed methods, including edge cases.
• Retain all functionality from prior work.

Specific Design Goals (may be expanded as project continues)

• Inspect implementation structure to refactor out duplicated code
• Review def self.reviewer_self_topic in review_bid for refacotring to address DRY violation
• Modify app/views/review_bids/_all_actions.html.erb as fitted to integrate reafactored controller modules
• Update tests to test the refactored DRYed-out methods

Previous Implementation VS New Implementation (may change as project continues)

The before diagram describes the current implementation of the review bidding system. Currently, the available topics are pulled from different controllers, depending on if the user is bidding for topics or topics to review. The new implementation involves created a single controller that will populate both the topic and review bidding features.

Expected File Change

Since the task is based on previous attempted implementational changes, lottery_controller.rb, review_bids_controller.rb, student_review_controller.rb, review_bids_helper.rb, review_bid.rb, show.html.erb, _topics.html.erb will be changed and used or added for implementation.

Test Plan

In the design planning phase of the project, most of the test plan will be focusing on manual testing that is highly concise for lightweight validation.

Once the design feature is further into develop, this page will be updated with specific code regarding the RSpec test cases desired for this feature. Until then, the following are outlined below:

Bidding Controller/View Tests

All previously created tests for the bidding functionality of topics must still pass.

Review Controller/View Tests

Unit Test Cases:

• A review bidding instance can be created by an instructor
• A review bidding instance cannot be created by a student
• A student can assign their preferences to a review bidding instance
• When the deadline for bidding passes, students are given their reviews
• (Edge Case) A Student cannot be choose to, nor be assigned to, review their own project.
• (Edge Case) When a student does not bid on a review when the deadline passes, they are assigned one randomly.
• (Edge Case) In a sufficiently large class, if no students bid on a review, that project can still be reviewed.

Feature Test Cases:

• A professor can see the option to make an assignment's reviews have the "Bidding" style and select it
• A student can see the project names of available reviews
• A student can drag multiple reviews into their review preferences
• When multiple students wish to be given the same review, the number of students assigned to a review does not exceed the maximum allowed
• Under a specific combination of choices of preference, the algorithm assigns students to the reviews they should theoretically receive

Github

The Github corresponding to this task is publicly available here: https://github.com/WeiRui-Wang/expertiza

Contributors

This feature was created as part of Dr. Edward Gehringer's "CSC/ECE 517: Object-Oriented Design and Development" class, Fall 2021. The contributors were: Geoff Garrido, Ayush Luthra, and WeiRui Wang. Our project mentor was Luis Delossantos (lgdeloss@ncsu.edu)