CSC/ECE 517 Fall 2020 - E2085. Allow reviewers to bid on what to review: Difference between revisions
(→Design) |
|||
| Line 29: | Line 29: | ||
===''Potential Challenges''=== | ===''Potential Challenges''=== | ||
=== ''How it works currently'' === | |||
It is important to understand how the current functionality works to plan for improving the functionality. | |||
Currently, After submitting a project/assignment, the student might also have a requirement to review others work. In order to perform a review, the student first selects some of the topics which are not yet assigned to 10 other participants. If the student chooses he "I do not care about the topic" checkbox, a random topic is picked from the existing topics and is assigned to the student. | |||
In other words, the student can either pick a topic or is randomly assigned a topic. There is no algorithm that does the selection. In this scenario, it is possible that some good topics are already picked by a lot of students and it someone is late to do the selecting, they miss out on it. | |||
===''Proposed methodology for matching students and topics''=== | |||
To solve the above problem of not getting the topics they like, the participants must be assigned topics based on a list of topics they pre-select. | |||
This is where an algorithm will come in handy. For picking a suitable algorithm, we should first identify the type of the problem. | |||
<br> | |||
<b>Type of problem</b><br> | |||
Each student should be able to bid on multiple assignments of their choice and the same assignment can be assigned to multiple participants for review. | |||
[[File:many-to-many.jpg]] | |||
Revision as of 02:13, 22 October 2020
Project Overview
Background
Assigning reviews to users is a complicated process. Currently, reviews are still assigned using a first-come-first-served basis. However, reviews could be bid on like how topics are bid on. This would involve matching multiple students to review a submission up to the maximum reviewers for a submission.
Objectives
- Implement top trading cycles on a web service.
- Add front end code to allow bidding on topics and call the appropriate web service from the lottery controller.
- Add test cases regarding new changes
Previous Work
There have been two previous implementations:
Relevant Links
Github Repository: https://github.com/uahamedncsu/expertiza/tree/beta
Pull Request: https://github.com/expertiza/expertiza/pull/1822
Video Demo: [Not Created]
Files Modified
Will list all the files we change while implementing our project
Design
Here we talk about the design plan and include figures and whatever else we need.
User Interface
Potential Challenges
How it works currently
It is important to understand how the current functionality works to plan for improving the functionality. Currently, After submitting a project/assignment, the student might also have a requirement to review others work. In order to perform a review, the student first selects some of the topics which are not yet assigned to 10 other participants. If the student chooses he "I do not care about the topic" checkbox, a random topic is picked from the existing topics and is assigned to the student. In other words, the student can either pick a topic or is randomly assigned a topic. There is no algorithm that does the selection. In this scenario, it is possible that some good topics are already picked by a lot of students and it someone is late to do the selecting, they miss out on it.
Proposed methodology for matching students and topics
To solve the above problem of not getting the topics they like, the participants must be assigned topics based on a list of topics they pre-select.
This is where an algorithm will come in handy. For picking a suitable algorithm, we should first identify the type of the problem.
Type of problem
Each student should be able to bid on multiple assignments of their choice and the same assignment can be assigned to multiple participants for review.
Implementation
Once we begin working, this section will have all our changes and be categorized by tasks
Test Plan
This section outlines the test plan for this project including testing scenarios/edge cases, the manual testing plan, and the automatic/RSpec testing plan.
Basic Test Scenario
- Reviewers bid on multiple projects at different times
- Bidding algorithm should give priority to reviewers that bid first and should assign based on reviewers ranking
Edge Cases
- All reviewers bid on the same project
- If different time stamps: bidding algorithm should give priority to reviewers that bid first
- If same time stamps: bidding algorithm should assign reviews randomly
- None of the reviews bid on any projects
- Bidding algorithm should assign reviews randomly
- Reviewer bids on their own project
- Bidding algorithm should not allow a reviewer to bid on their own project and should have validation to prevent assigning a reviewer their own project
- None of the reviewers bid on a specific project
- Bidding algorithm should still assign reviewers to this project
Manual Testing Plan
Manual testing should be preformed on an Expertiza server.
Manual testing should (at a minimum):
- test that the UI is implemented as expected
- test that a participant can bid on projects to review
- test that a participant can move projects into the bidding column
- test that a participant can reorder projects in the bidding column
- test that a participant's bidding preferences save
RSpec Testing Plan
RSpec tests will need to be written to related controller(s) and model(s).
These RSpec tests should (at a minimum):
- test basic functionality of controller(s)
- test validation of the model(s)
- test the scenarios (basic case & edge cases) mentioned above
Team
Yasmin Ahamed Adam (uahamed)
Ryan Grainger (rpgraing)
Luis Delossantos (lgdeloss)
Colleen "Bria" Engen (ceengen)
Mentor: Saurabh Shingte (svshingt)
Resources
Previous Implementations: [1], [2]
Bidding Interface Implementation Pull Request: [3]