CSC/ECE 517 Fall 2021 - E2148. Completion/Progress View
Problem Statement
In Expertiza, peer reviews are used as a metric to evaluate someone’s project. Once someone has peer-reviewed a project, the authors of the project can also provide feedback for this review, called “author feedback.” While grading peer reviews, it would be nice for the instructors to include the author's feedback, since it shows how helpful the peer review actually was to the author of the project. Currently, however, the instructor has no easy way of seeing the author-feedback scores, so it would be far too much trouble to include them in grades for reviewing. The aim of this project is to build this into the system.
Breaking down the problem statement into 3 subproblems:
1. A toggle button to view/hide average authors scores
The review report is already pretty crowded. The addition of a new column for average authors' feedback scores will make things clumsy. And often there is no author feedback, so it will be mostly wasted space. So, we are updating the UI in such a way that it's up to the instructor to view or hide the author's feedback. So, a toggle button at the top of the table will be added to view or hide the average author's feedback. When average authors' feedback is viewed the reviewer column will be squeezed, by using wrapping, in order to allocate space to the average authors' feedback column.
Old UI
LOFI UI sketch
2. Author's summary link
Right now, if the average author's feedback is too low, there is no way for the instructor to take a look at the author's comments. It's good if we can maintain an author's summary page where the instructor can take a look at the author's comments in order to come to a conclusion about the low average author's feedback. Sometimes even the reviewer's comments can be attacking the author's work. So if the author expresses such kinds of issues in the author's feedback instructor will be aware of how legitimate and polite the reviewer was if we can expose the author's comments somewhere in the system. The plan is to add this link in the reviews done column right after the summary link.
LOFI UI sketch
3. Austor's summary page
As discussed in the 2nd point, a new author's summary link will be provided on the review report page which will redirect the instructor to a new author's summary page. The author's summary page will have all the author's comments for a student who has reviewed the project. Each team author's scores will be rendered one by one.
LOFI UI sketch
Where to find review report UI?
When logged in as an instructor, click on Manage -> Assignments. You'll be able to see the list of all assignments available. For a particular assignment, click on the view reports icon (search image on the head of a person) which is just beside the view scores icon (start icon with search image). Then select Review report from the drop-down and click on the view button.
http://152.7.98.254:8084/reports/response_report?id=834 - temporary link to navigate to the review report UI page. Select Review report from the drop-down and click on the view button.
Scope
Though the project mentioned 5 points that need to be done as part of this project, there are a few features that are already built into the system.
Integration of review performance – Something that combines the author feedback data with the review data so instructors can grade reviewers based on:
1. # of reviews completed.
2. Length of reviews.
3. [Summary of reviews].
4. Whether reviewers added a file or link to their review.
5. The average ratings they received from the authors.
1. # of reviews completed. - This is already present in the current UI. Reviews Done column in review report page gives the number of reviews completed.
2. Length of the reviews. - This is already present in the current UI, but in the form of metrics. The metrics column on the review report page gives a visual representation of what's the average length of reviews given by the reviewer Vs what's the average length of reviews given by the class.
3. Summary of reviews. - In the reviews done column a summary link is available which will navigate the instructor to the summary page where the instructor can look at the summary of comments given by the reviewer.
The scope of the present project is just to focus on the last 2 points, i.e., the Instructor should be able to know whether reviewers added a file or link to their review and The average ratings reviewer received from the authors.
Design
1. A toggle button to view/hide average authors scores
First, in order to add the average author's scores we need to calculate the average author's scores and render those scores in the review report UI.
Overview on how to calculate the average author's scores for each team.
The questions table has all the questions. For our use case, we will be considering only the feedback questions. Answers and scores to these questions will be saved in the Answers table with question id being the foreign key. Now, the tricky part is to know whether the answer is feedback given by an author or the review given by the reviewer we will make use of the Response Map table. Response id column in the Answers table will map to the id Response table with response id being the foreign key in the Answers table. The map_id column in the Response table will map to the id column in the Response Map table. The type column in the Response Map table will give us the info on whether a particular answer belongs to feedback or not. If the type of a particular answer is FeedbackResponseMap then that is feedback given by the author. For each team, the average of all authors' feedback given will be calculated which can be obtained from the Answers table.
Table Structures
Files being edited as part of this change.
1. https://github.com/VarunVeginati/expertiza/blob/master/app/models/on_the_fly_calc.rb - to add calculation related to average author's feedback.
2. https://github.com/VarunVeginati/expertiza/blob/master/app/helpers/report_formatter_helper.rb - to store the calculated average author's feedback as a class variable in order to access it in the corresponding HTML file. The class variable will be assigned in the review_response_map method since this method will be called when a user opens the review report page.
3. https://github.com/VarunVeginati/expertiza/tree/master/app/views/reports - A new HTML file called _teammate_feedback_report.html.erb will be created in order to render each row in author's feedback column.
4. https://github.com/VarunVeginati/expertiza/blob/master/app/views/reports/_review_report.html.erb - This is the actual HTML page where the review report is rendered. A new column and a toggle button to view/hide the author's feedback will be added in this HTML file.
Testing
UI testing will be done manually. We will use Rspec testing for the author feedback score functions used for computing author scores and averages.