CSC/ECE 517 Fall 2021 - E2168. Testing - Reputations
Project Overview
Objectives
Our objectives for this project are the following:
- Double and stub an assignment, a few submissions to the assignment, under different review rubrics
- Manually calculate reputation scores based on paper "Pluggable reputation systems for peer review: A web-service approach"
- Validate correct reputation scores based on different review rubrics generated by reputation management VS manual computation of reputation score expectation
Files Involved
- reputation_web_service_controller_spec
- hamer_spec
lauw_spec (?)
Test Plan
Testing Objects
Following objects will be created and confined for the purpose of testing.
- assignments
- submission_records
- questionnaires
- assignment_questionnaires
- participants
- teams
- teams_users
- users
- responses
- response_maps
Relevant Methods
hamer_spec
- calculate_weighted_scores_and_reputation
This is the main method of Hamer algorithm. Reviewers inaccuracy will be calculated and updated from past average, then weight of the review will be adjusted based on the inaccuracy. We plan to create two contexts to test it as below: 1. When the review is the same from average, reviewer’s inaccuracy should be minimized and the review’s weight should be maximized; 2. When the review is furthest from average, reviewer’s inaccuracy should be maximized and the review’s weight should be minimized.
- converged?
This method ensures the final result of calculated reviews' weights is converged, as the algorithm requires. A simple fixture test can be done by giving the method a set of input data and compare it to expected result.
ReputationController_spec
- db_query
This is the normal db query method, call this method will return peer review grades with given assignment id. We will test this method in two aspect. 1. Test whether or not the grade return is right based on the specified algorithm. 2. We need to test the correctness of the query.
- db_query_with_quiz_score
This is the special db query, call this method will return quiz scores with given assignment id. We will test this method with same logic as the first one.
- json_generator
This method will generate the hash format of the review, we will test this method by calling to and convert the result to json format the print to test its correctness.
- client
client method will fill many instance variables with corresponding class variables, we need to test this method with send_post_request.
- send_post_request
This method send a post request to peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms to calculate get the reputation result and use show the result in corresponding UI and update given reviewer's reputation. We will test this method based on the algorithm in the paper, first to test the result reputation value, second to test the update value in database.
Results
Collaborators
Jinku Cui (jcui23)
Henry Chen (hchen34)
Dong Li (dli35)
Zijun Lu (zlu5)