CSC/ECE 517 Fall 2021 - E2168. Testing - Reputations
Project Overview
Objectives
Our objectives for this project are the following:
- Double and stub an assignment, a few submissions to the assignment, under different review rubrics
- Manually calculate reputation scores based on paper "Pluggable reputation systems for peer review: A web-service approach"
- Validate correct reputation scores based on different review rubrics generated by reputation management VS manual computation of reputation score expectation
Files Involved
- reputation_web_service_controller_spec
- hamer_spec
lauw_spec (?)
Test Plan
Testing Objects
Following objects will be created and confined for the purpose of testing.
- assignments
- submission_records
- questionnaires
- assignment_questionnaires
- participants
- teams
- teams_users
- users
- responses
- response_maps
Relevant Methods
hamer_spec
- calculate_weighted_scores_and_reputation
This is the main method of Hamer algorithm. Reviewers inaccuracy will be calculated and updated from past average, then weight of the review will be adjusted based on the inaccuracy. We plan to create two contexts to test it as below: 1. When the review is the same from average, reviewer’s inaccuracy should be minimized and the review’s weight should be maximized; 2. When the review is furthest from average, reviewer’s inaccuracy should be maximized and the review’s weight should be minimized.
converged?
This method ensures the final result of calculated reviews' weights is converged, as the algorithm requires. No testing is needed on this method.
ReputationController_spec
- db_query
- db_query_with_quiz_score
- json_generator
- client
- send_post_request
Results
Collaborators
Jinku Cui (jcui23)
Henry Chen (hchen34)
Dong Li (dli35)
Zijun Lu (zlu5)