CSC/ECE 517 Fall 2017/E17A7 Allow Reviewers to bid on what to review

Introduction

Description of project

Expertiza provides a way for students to bid on different topics for proposed assignment topics. This allows the fair topic distribution among students when multiple students or groups are bidding on same topic or paper.

The changes related to this project are required for reviewing conference papers and journals. The users are able to submit their works as either individuals or teams to be scrutinized. The other participants are allowed to review the papers of their interest from the list of submitted papers. The participants are allowed to bid on the papers of their preference in a preference list with the top most entry taking the highest priority. The bidding algorithm already exists as called by a web service in Peerlogic. The goal of this project is to generalize the algorithm for both review and topic based bidding.

Bidding Algorithm Overview

Currently, the Bidding algorithm implementation is based on k-means clustering and a weighting formula that favors increasing overall student satisfaction and adding members until the maximum allowable team size is reached. The approach to meeting these criteria is addressed by mining student preferences for topics with a clustering approach, and then matching them in groups to topics that suit their shared interests. You can read about it in detail here.

Problem Statement

• Reviews are often assigned by having reviewers bid on papers they want to review. This is very similar to the kind of bidding that Expertiza implements for topics, and it should be possible to call the same code (which will be modified to make it more general).

• The reviewer bidding for proposed conference papers has not been fully implemented. Currently the bidding process for assigning topics is either manual or automatic. Manual means the instructor assigns topics based on the preference provided by the students. Automatic assignment means that topics are assigned on the basis of an implemented algorithm, which takes into account some parameters, including intelligent-property.

Our goal here is as follows:

• To implement a bidding algorithm for conference paper reviewer assignment functionality.
• To attempt to merge the existing and the new algorithm as much as possible

Possible Extension

A possible extension is to combine reviewer assignment by bidding with one of the other assignment methods--instructor assignment, or “automated” assignment, where each new reviewer is assigned one of the least-reviewed submissions so far. In this case, submissions that are not bid upon would be assigned to reviewers who “show up later,” as would reviews that are not completed by the reviewers who bid on them.

Use Case Diagram

Files to be Modified

• lottery_controller.rb
• New Controller to be specified for review bidding

Test Plan

Manual Testing

• UI testing of the implemented functionality to be done.

Automated Test Cases

• TDD and Feature Test cases to be written.

Edge cases

• Case 1: Users signing up after bidding is done need to be assigned the papers/journals with no or least reviews done.

• Case 2: Extending edge case 1, if somebody doesn't provide a list of topics s(he) wants to review, the topic with the least number of bids is assigned.

• Case 3: If a participant is assigned to review a paper in the (topics) preference list but decides to drop the review after the bidding process is done.