CSC/ECE 517 Spring 2020 - E2021. Allow reviewers to bid on what to review
Motivation
Currently, the projects to review are assigned to the reviewers on a first come first serve basis. This policy does not let the reviewers investigate and learn from the projects they are specifically interested in. Therefore, the idea here is to let the students decide on what projects they wish to review in a fair manner by placing a bidding policy.
Problem Statement
The reviews performed by the students right now are being assigned on a first-come-first-serve basis. The task at hand is to implement a procedure so that the students can bid on the reviews that they would like to review, similar to the procedure of the assignment of the topics. The difference between the two procedures is that the assignment of the topic is a one-one mapping, i.e. a team is assigned to a single topic, whereas the assignment of the reviews is many-many mapping with each review being assigned to different students and each student assigned to different reviews.
The need for a bidding system for reviews
The bidding process, when implemented for reviews of projects, will hold several advantages.
1) The users will be able to bid for reviews that interest them. This will incentivize them to get a better understanding of the project requirements and provide the necessary feedback in an informed manner.
2) Bidding provides the users receiving feedback a very informed report on how to enhance their project. This will help them get better results for their respective projects.
3) The bidding also gives an advantage to the instructor because the instructor will now be able to receive more well-written, useful and meaningful feedback that will be more useful in their academic projects.
What needs to be done
The following are the tasks that are to be done-
1. Implement top trading cycles on a web service as mentioned in the paper [1].
The top trading cycles is an intelligent algorithm that allows users to achieve the review they will need. For example, let us consider a situation where user 'A' wants the review assigned to user 'B' and user 'B' wants the review assigned to user 'A'. Considering that it will motivate the students if they get the project that they would like to review, this mutual exchange between them would be beneficial. It can be solved using the top trading cycles algorithm.
Algorithm Steps
1) Let the user bid, these bids will point to the reviews they are requesting for in the graph.
2) There will be cycles links of length 1 between the person who owns ever review currently, a directed acyclic graph.
3) We will need to break all the cycles in the graph and assign the reviews to which the user is pointing at. The top is represent as [ TOP(i) ]
4) We can repeat this order until we have exhausted our preference list where all the assigned houses are constantly deleted.
From the diagram we can see that the three users are represented as u1, u2, u3 and the the three reviews are r1,r2,r3.
In phase 1 : all users want the review 1 so all of them point to r1 and we see r1 is owned by u1 and r2 is owned by u2 and r3 by u3.
In phase 2 : u1 is assigned r1 and now u2 points to r3 which is now its highest priority because r1 is removed and similarly u3 points to r2.
In phase 3: r2 is assigned to u3 and and r3 is assigned to u2, thus satisfying all.
2. Add the front end code to allow the bidding on topics and call the appropriate web services from the lottery controller. This is similar to the already implemented bidding process on topics [2].
flow chart for this us functionality would involve :
Previous Work
The latest implementation of the problem was E1986 [3][4].
From the previous implementations we can see that the following files were modified in order to get the required implementation they have modified the following files :
Modified Files
routes.rb app/views/student_task/view.html.erb app/views/assignments/edit.html.erb app/controllers/student_task_controller
New Files
app/controllers/review_bidding.rb app/models/review_bidding_controller.rb app/views/sign_up_sheet/review_bid.html.erb Source Code for the Web Service [1]
Problems found in previous work
The problems in the previous work were given-
1. The link to the run bidding assignment is on the Review Strategy tab instead of the actions associated with an assignment like the topic assignment. This is an inconsistency in the UI that needs to be remedied.
2. All of the implementation code was put in the controller which is inappropriate for a controller. The code should probably be in the corresponding model file.
3. The tests that they have written are not adequate.
Suggested Improvements
References
[1] School Choice: A Mechanism Design Approach by Atila Abdulkadiroğlu and Tayfun Sönmez.
[2] Bidding Interface Implementation.
[3] E1986 Pull Request.
[4] E1986 Wiki page.