CSC/ECE 517 Fall 2020 - E2085. Allow reviewers to bid on what to review

Project Overview

Background

Assigning reviews to users is a complicated process. Currently, reviews are still assigned using a first-come-first-served basis. However, reviews could be bid on like how topics are bid on. This would involve matching multiple students to review a submission up to the maximum reviewers for a submission.

Objectives

1. Implement top trading cycles on a web service.
2. Add front end code to allow bidding on topics and call the appropriate web service from the lottery controller.
3. Add test cases regarding new changes

Previous Work

There have been two previous implementations:

• Fall 2019 here
• Spring 2020 here

Relevant Links

Github Repository: https://github.com/uahamedncsu/expertiza/tree/beta
Pull Request: https://github.com/expertiza/expertiza/pull/1822
Video Demo: [Not Created]

Files Modified

Will list all the files we change while implementing our project

Design

Here we talk about the design plan and include figures and whatever else we need.

How it works currently

It is important to understand how the current functionality works to plan for improving the functionality. Currently, After submitting a project/assignment, the student might also have a requirement to review others work. In order to perform a review, the student first selects some of the topics which are not yet assigned to 10 other participants. If the student chooses he "I do not care about the topic" checkbox, a random topic is picked from the existing topics and is assigned to the student. In other words, the student can either pick a topic or is randomly assigned a topic. There is no algorithm that does the selection. In this scenario, it is possible that some good topics are already picked by a lot of students and it someone is late to do the selecting, they miss out on it.

Analyzing the problem

To solve the above problem of not getting the topics they like, the participants must be assigned topics based on a list of topics they pre-select. This is where an algorithm will come in handy. For picking a suitable algorithm, we should first identify the type of the problem.
Type of problem
Each student should be able to bid on multiple assignments of their choice and the same assignment can be assigned to multiple participants for review.

<<image>>

The algorithm of choice for this type of stable allocations is Gale-Shapely algorithm. An unstable allocation is when the participant does not like the topic assigned to them. This type of problem where we need to find a stable matching between two equally sized sets of elements given an ordering of preferences for each element is called a Stable marriage problem.
In the real world, there are cases when we face this problem:

• Student choosing their future universities School Choice Problem (One-to-Many)
• Men and women seeking each other to create families Stable Marriage Problem (One-to-One)
• Internet services that need to be connected with users with the smallest amount of time

Proposed methodology for matching students and topics

The idea of the Algorithm is to iterate through all free topics. Every free topic goes to a participant who selected the topic based on preference on the participants list. If the topic is not already assigned to someone or if it is assigned to only a limited number of students less than the threshold of 4, then the topic is assigned to the participant as a temporary pairing.

<< section in progress.. >>

The time complexity of this algorithm is O(n^2)

Implementation

Once we begin working, this section will have all our changes and be categorized by tasks

Test Plan

This section outlines the test plan for this project including testing scenarios/edge cases, the manual testing plan, and the automatic/RSpec testing plan.

Basic Test Scenario

• Reviewers bid on multiple projects at different times
  • Bidding algorithm should give priority to reviewers that bid first and should assign based on reviewers ranking

Edge Cases

• All reviewers bid on the same project
  • If different time stamps: bidding algorithm should give priority to reviewers that bid first
  • If same time stamps: bidding algorithm should assign reviews randomly
• None of the reviews bid on any projects
  • Bidding algorithm should assign reviews randomly
• Reviewer bids on their own project
  • Bidding algorithm should not allow a reviewer to bid on their own project and should have validation to prevent assigning a reviewer their own project
• None of the reviewers bid on a specific project
  • Bidding algorithm should still assign reviewers to this project

Manual Testing Plan

Manual testing should be preformed on an Expertiza server.
Manual testing should (at a minimum):

• test that the UI is implemented as expected
• test that a participant can bid on projects to review
  • test that a participant can move projects into the bidding column
  • test that a participant can reorder projects in the bidding column
  • test that a participant's bidding preferences save

RSpec Testing Plan

RSpec tests will need to be written to related controller(s) and model(s).
These RSpec tests should (at a minimum):

• test basic functionality of controller(s)
• test validation of the model(s)
• test the scenarios (basic case & edge cases) mentioned above

Team

Ummu Kolusum Yasmin Ahamed Adam (uahamed)
Ryan Grainger (rpgraing)
Luis Delossantos (lgdeloss)
Colleen "Bria" Engen (ceengen)
Mentor: Saurabh Shingte (svshingt)

Resources

Previous Implementations: [1], [2]
Bidding Interface Implementation Pull Request: [3]