CSC/ECE 517 Fall 2018/E1856 Allow reviewers to bid on what to review

Description

Students currently are able to “bid” for the projects that they want to do as an assignment. On the other hand, for reviewing others’ work, the policy that’s currently in use is called “first-come-first-serve”.

In this project, we update the “first-come-first-serve” policy for assigning projects to reviewers, to the “bidding” policy. In the backend, we implement the “top trading cycles” algorithm. And we modify the controller to meet the needs of the algorithm. At the front end, we create an UI design to present to the students their choices to select.

Introduction

The bidding policy for project topics that students want to work on is already implemented. Students can currently bid on project topics for there team project topics. This reduces a lot of possible conflicts in assigning topics to each team.

There’s also reviewing work for each assignment. Currently, the policy to assign the project to the reviewer is “first-come-first-serve”. Students that choose to review a project first will get that project.

However, this policy creates an issue - While reviewing students’ work, sometimes the same project is requested to be reviewed a lot of times, while some other projects are not requested as much. A similar bidding policy for assigning projects to reviewers can help students who want to review a project the most to be most likely to receive that project. The completion of this project will allow students to also bid on what projects they are interested in reviewing.

We need to alter the “first-come-first-serve” to the “bidding” policy. In this policy, students will be matched to review a submission up to the maximum reviewers.

The projects includes implementation of a top trading cycles algorithm on Expertiza. A UI design for the bidding front-end will also be developed for the end-users.

Current System Status

Regarding to the functionality of review mapping, it's necessary to describe some of the related models and controllers in expertiza and how they are organized to fully implement the functionality.

Models

1. When an assignment is released, an instance of Assignment is created and corresponding topic instances of SignUpTopics are also imported, indicating that different topics are available for students to choose from within this assignment.

2. Students are assigned as instances of AssignmentParticipant to this assignment and form up their teams(Model Team).

3. Each team registers for several topics and becomes a candidate instance of SignUpTeam. After topics bidding, some of the candidate teams become an official signed-up team of a particular topic.

4. After the team has submitted their response, a corresponding instance of Response is created.

5. Any participant selects any response except their own to review. A mapping between a signed-up team and the participant is created within this assignment.

Workflow

For now, expertiza employs FIFO strategy on review assignment. In the ReviewMappingController, method "add_reviewer" is invoked when students are trying to request a peer review. When it is done, a new mapping is created, that is why review assignment is in FIFO style.

However, topics are allowed to be bid via bidding interface. The LotteryController handles the topics bidding.

First of all, teams have signed up for several topics with different preference. For example, there are three topics within an assignment, namely A, B and C. Therefore 4 teams W, X, Y, Z. Teams would have their own preference towards different topics:

W: B, A, C | X: B, C | Y: C, A, B | Z: A, B, C

This data structure is passed to backend service http://peerlogic.csc.ncsu.edu/intelligent_assignment/merge_teams by method "run_intelligent_assignment" of LotteryController. The peerlogic service runs top_trading_cycles algorithm to have a fair bidding over the topics and teams.

Design

Almost all of the functionality for our project has already been implemented in the review portions of the Expertiza system. Because of this, we will approach this project by first using a decorator pattern to add lottery capability to the review mapping controller for choosing topics. Then we will duplicate the view for choosing project topics and modify it so that it interacts with the review mapping controller. We understand that this is not DRY code, so we will try to address this problem by seeing if there is any way we can modify the choose project topics page in order to reuse its code, yet still fulfill the functionality of review mapping.

Design Patterns

Model View Controller - Expertiza uses the Ruby on Rails framework which is set up to use the Model View Controller software pattern.

DRY Principle - We will be attempting to write our code in a dry way, that being that code within the system is not duplicated.