CSC/ECE 517 Fall 2019 - E1996. Enhancements to review grader

Background

    When the staff grade reviews, we look at them in the review-grader interface. The staff assign the score and provide feedback using the textboxes in the right column. The user are also allowed to read the reviews in the summary view. Some sections on these two page don't show correctly and some sections are not good-looking enough. The space can be rearranged to use the space more effectively. Implementing these enhancements to review grader will greatly improve the user experience of expertiza.

Problem Statement

The Main review-grader Page

Issue Description

1. The "Score awarded/average score" column is supposed to report the score by the current student reviewer in the first round, the average score by all reviewers in the first round, and scores for the second round. If the number of rounds ≠ 2, then the number of scores should be adjusted appropriately. Fix this problem so that it is populated with the correct numbers.
2. The scores in the "Team Reviewed" column would also be displayed with bar graphs, like in the "Metrics" column. It should be modified without making the column much wider, since this page needs to display a lot of information horizontally.
3. Column width should be adjusted intelligently. In the view shown, "Reviewer", "Reviews done", and "Team reviewed" are too wide, whereas "Assign grade and write comments" is too narrow.
4. Number the rows of the table (e.g., "2. Student 8370") so it is easy to count the lines. This will help us assign each TA (and the instructor) an equal number of reviews to grade.
5. Replace the team names with an anonymized version of them. This will help the grader not be biased if (s)he recognizes the team name as belonging to specific students.

Design Strategy

1. The score which user can see on the website is "-----", which is in the get_each_round_score_awarded_for_review_report function in the review_mapping helper file. The problem is that the review_scores[reviewer_ID] doesn't exist, which should be calculated in the file on_the_fly_calc.rb. There is something wrong with variable assignment in several methods. To solve the issue, assign the value to review scores after calculating in the on_the_fly_calc.rb.
2. The metrics feature has been implemented by display_volume_metric_chart function in the _review_report.html.erb file. What need to be done is to get the correct data and use that method to generate bar graph.
3. The width of column is set up in the _review_report.html.erb file from line 37 to line 43. Change the percentage distributed to every column to adjust the width.
4. In the _review_report.html.erb file, use method "each_with_index" instead of "each" to get the index of each loop. Then, add "index+1" before the reviewer name since the index starts from 0.
5. In the _review_report.html.erb file, replace the "team_reviewed_link_name" with randomly generated team names.

Design Pattern

This project mainly focuses on the UI, so no design pattern is needed.

Files to be modified

/app/models/on_the_fly_calc.rb
/app/views/reports/_review_report.html.erb

Test Plan

1. Login as instructor.
2. Go to "Manage >> Assignment" and choose one assignment with reviews like "Final project (and design doc)", click on "View Report".
3. Click "View". Here is the "The Main review-grader Page" mentioned above.
4. Check the "Score awarded/average score" column to see if the scores are displayed.
5. Check the "Reviewer", "Reviews done", "Team reviewed", and "Assign grade and write comments" columns to see if they are adjusted as intended.
6. Check the "Reviewer" column to see if there is a index in each row.
7. Check the "Team reviewed" column to see if the team names are anonymized.

The summary Page

Issue Description

1. Replace the reviewer name and team names with an anonymized version of them, as done on the main review-grader page.
2. Checkbox items take up far too much space. Remove duplicated header lines and show just columns of checkboxes to the right of the “questions.” Also, it is unnecessary to prefix each by “[Question]”.
3. Adjust column width intelligently. The “Reviewee” and “Score” columns are much wider than necessary. The “Comments” column is also too wide for easy reading. So consider how the page might be reorganized to take better advantage of the available space. One option might be to show the comments vertically, astride each other, rather than horizontally, above and below each other. Mock up your proposal and discuss it with your mentor.
4. Get rid of “Review: Round1”. It should be, “Review Round 1”

Design Strategy

1. For the reviewer name, replace the "Participant.find(@reviewer_id).fullname" with "Participant.find(@reviewer_id).name" in the view_review_scores_popup.html.erb file. For the team names, replace the "team.name" with randomly generated team names.
2. To get rid of the duplicated header lines, we will modify view_review_scores_popup.html.erb. Change the view so that only the checks are presented to the right of the questions. Also delete the “[Question]” before each questions. Below is the mockup table we are planning to implement. In this way, we won't have repeated headers and the constructure of comment questions and checkbox questions are uniformed.
   
3. To adjust column width intelligently, we will change the preset width in /app/views/popup/view_review_scores_popup.html.erb. 10% for "Reviewee", 5% for "Score" may be reasonable. Also we will try two comments on same role or show the comments vertically and see which way is better.
4. We located the "Review Round" header in /app/models/review_response_map.rb.

Files to be modified

/app/views/popup/view_review_scores_popup.html.erb
/app/models/review_response_map.rb
/app/assets/stylesheets/grades.scss
/app/views/popup/_checkbox_question_review.html.erb(file added)
/app/views/popup/_non_checkbox_question_review.html.erb(file added)

Test Plan

1. Login as instructor.
2. Go to "Mange >> Assignment" and choose one assignment with reviews like "Final project (and design doc)", click on "View Report".
3. Click "View". Here is the "The Main review-grader Page" mentioned above.
4. Click one of the "Summary", and here is "The summary Page" mentioned above.
5. Check if the reviewer names and team names are all anonymized.
6. Check if the checkbox items are displayed as expected.
7. Check if column widths are appropriate..
8. Check if “Review: Round1” is changed to “Review Round 1”.

Modification

The modification are mainly on 'Main review-grader' Page and 'Summary' page. Comparisons are shown below.

The Main review-grader Page

1. Fix the “Score awarded/average score” column so that it is populated with the correct numbers.
   For the "Score awarded", we add "@review_scores[response_map.reviewer_id] = reviewer" at the end of mothod "scores_varying_rubrics" in file "app/models/on_the_fly_calc.rb" to solve the problem.
   For the "average score", we didn't fix the bug. But we have figured out where the problem is and the progress of calculating average scores. The problem only occurs in the first round review for some teams. First, in file "app/views/reports/_team_score.html.erb", it use "@avg" to show average scores. The "@avg" is assigned in method "get_review_metrics" in file "app/helpers/review_mapping_help.rb". It is set to "-----" at the beginning of the method. If "@avg_and_ranges[team_id][round][metric]" is not nil, the "@avg" will be set to the average scores. Otherwise, it will still be "-----". The "@avg_and_ranges" is set to the result returned by method "compute_avg_and_ranges_hash" in method "review_response_map" in file "app/helpers/report_formatter_helper.rb". The method "compute_avg_and_ranges_hash" is defined in file "app/models/on_the_fly_calc.rb". In method "compute_avg_and_ranges_hash", it calls method "compute_scores" in file "app/models/answer.rb" with assessments for a certain team and questions as arguments. In the method "compute_scores", it checks if the assessments passed to the method are present. If the assessments are not present, average scores will be set to nil. So, the reason why average scores of some teams are shown as "-----" is that the assessments(reviews) for these teams don't exist. But these teams have review scores from some reviewers which means these teams should have reviews for them. During the limited time we didn't figure out the reason why these teams have review scores from reviewers but the reviews don't exist. This problem is more complicated than it seems.
   Before Modification:
   
   After Modification:
   
   
2. Column width should be adjusted intelligently. In the view shown, "Reviewer", "Reviews done", and "Team reviewed" are too wide, whereas "Assign grade and write comments" is too narrow.
   Before Modification:
   
   After Modification:
   
3. Numbered the rows of the table (e.g., "2. Student 8370") so it is easy to count the lines. Also removed the duplicated student name in brackets. This will help us assign each TA (and the instructor) an equal number of reviews to grade.
   In file "app/views/_review_report.html.erb", we used method "each_with_index" instead of "each" and add "index+1" before reviewer names.
   Before Modification:
   
   After Modification:
   
4. Replace the team names with an anonymized version of them. This will help the grader not be biased if (s)he recognizes the team name as belonging to specific students.
   In file "app/models/team.rb", we added methods called "self.anonymized_view?" and "name" which is pretty similar as those in file "app/models/user.rb" to switch the team name to anonymized one. The methods take an IP argument to see if anonymized mode is turned on. If it is turned on, the anonymized name will be used. Otherwise, the original name will be shown.
   Before Modification:
   
   After Modification:
   
   
5. Adjust the bar chart. Rearrange elements, adjust the weight of bar, direction, etc and fix bugs. Remove the legend from chart(yellow and red circles) and put the legend into table head.
   Before Modification:
   
   After Modification:
   
6. Adjust the size of text are in the comments column(last column), improve user experience.
   Before Modification:
   
   After Modification:
   
   

The summary Page

1. Replace the name of the reviewer with the anonymized version, as done on the main review-grader page.
   In file "app/popup/view_review_scores_popup.html.erb", use Participant.find(@reviewer_id).name(session[:ip]) instead of "Participant.find(@reviewer_id).name" to pass argument to the method "name". Therefore, the method can see if the anonymized mode is turned on and do appropriate action.
   Before Modification:
   
   After Modification:
   
   
2. The code can be located in /app/views/popup/view_review_scores_popup.html.erb. In the origin code, checkbox and question are in two different row, which takes too much space. We changed the structure of the code to make these two section into different cells in one row. Then we modified the style of the table to rearrange the width of cell. For the duplicated team name, we took the header out of the "question.each" loop and put it at the top of the table as a table header. We also removed the "[question]" text, which seems a little bit redundant. To make the code more readable, we distributed the code of checkbox question and non-checkbox question into two different files after all the changes.
   Before Modification:
   
   After Modification:
   
3. At first we removed the duplicated team name and reset the width of column. This can be implemented simply by change the width percentage of the column. These changes look good but they doesn't fit with the whole page. To keep consistent with checkbox question, we put the non-checkbox question to the left of the answer as well, which can be approached by the same method as issue two. After that we found that one independent cell for scores took too much space. We put the scores into the "question.each" loop of answer to save more space, and we add one grade circle style in file app/assets/stylesheets/grades.scss to make the scores more good-looking.
   Before Modification:
   
   After Modification:
   
4. We implemented our design by add a blank into the prepare_review_response function in review_response_map.rb.
   Before Modification:
   
   After Modification:
   

Test

1. In file "spec/models/on_the_fly_calc_spec.rb", change "expect(assignment.compute_reviews_hash).to eql({})" to "expect(assignment.compute_reviews_hash).to eql({1=>{1=>{1=>50}}, 2=>{1=>{1=>30}}})". This test is for the score from reviewers.
2. In file "spec/models/review_response_map_spec.rb", change ".to eq("review round1": {questionnaire_id: 1, response_ids: [1]}, "review round2": {questionnaire_id: 1, response_ids: [2]})" to ".to eq("review round 1": {questionnaire_id: 1, response_ids: [1]}, "review round 2": {questionnaire_id: 1, response_ids: [2]})". This test is for the change that changes "Review Round1" to "Review Round 1".