CSC/ECE 517 Fall 2021 - E2148. Completion/Progress View
Problem Statement
In Expertiza, peer reviews are used as a metric to evaluate someone’s project. Once someone has peer-reviewed a project, the authors of the project can also provide feedback for this review, called “author feedback.” While grading peer reviews, it would be nice for the instructors to include the author's feedback, since it shows how helpful the peer review actually was to the author of the project. Currently, however, the instructor has no easy way of seeing the author-feedback scores, so it would be far too much trouble to include them in grades for reviewing. The aim of this project is to build this into the system.
Breaking down the problem statement into 3 subproblems:
1. A new column and a toggle button to view/hide average authors scores
A new column called AVG feedback scores is added. But the review report is already pretty crowded. The addition of a new column for average authors' feedback scores will make things clumsy. And often there is no author feedback, so it will be mostly wasted space. So, we updated the UI in such a way that it's up to the instructor to view or hide the author's feedback. So, a toggle button at the top of the table is added to view or hide the average author's feedback.
Old UI
New UI
2. Author's summary link
Right now, if the average author's feedback is too low, there is no way for the instructor to take a look at the author's comments. It's good if we can maintain an author's summary page where the instructor can take a look at the author's comments in order to come to a conclusion about the low average author feedback scores. Sometimes even the reviewer's comments can be attacking the author's work. So if the author expresses such kinds of issues in the author's feedback, the instructor will be aware of how legitimate and polite the reviewer was if the author's comments were exposed somewhere in the system. The new authors summary hyperlink will solve this issue.
New UI having authors summary hyper link
3. Author's summary page
As discussed in the 2nd point, a new author's summary is provided on the review report page which will redirect the instructor to a new author's summary page. The author's summary page will have all the author's comments for a student who has reviewed the project along with the score for each comment. The author's summary page will be rendered only when the instructor clicks on the link.
New UI page for author's summary
Where to find review report UI?
When logged in as an instructor, click on Manage -> Assignments. You'll be able to see the list of all assignments available. For a particular assignment, click on the view reports icon (search image on the head of a person) which is just beside the view scores icon (start icon with search image). Then select Review report from the drop-down and click on the view button.
http://152.7.99.72:8086/reports/response_report?class=form-inline&id=834 - temporary link to navigate to the review report UI page. Select Review report from the drop-down and click on the view button.
Scope
Though the project mentioned 5 points that need to be done as part of this project, there are a few features that are already built into the system.
Integration of review performance – Something that combines the author feedback data with the review data so instructors can grade reviewers based on:
1. # of reviews completed.
2. Length of reviews.
3. [Summary of reviews].
4. Whether reviewers added a file or link to their review.
5. The average ratings they received from the authors.
1. # of reviews completed. - This is already present in the current UI. Reviews Done column in review report page gives the number of reviews completed.
2. Length of the reviews. - This is already present in the current UI, but in the form of metrics. The metrics column on the review report page gives a visual representation of what's the average length of reviews given by the reviewer Vs what's the average length of reviews given by the class.
3. Summary of reviews. - In the reviews done column a summary link is available which will navigate the instructor to the summary page where the instructor can look at the summary of comments given by the reviewer.
The scope of the present project is just to focus on the last 2 points, i.e., the Instructor should be able to know whether reviewers added a file or link to their review and The average ratings reviewer received from the authors.
Design
1. A toggle button to view/hide average authors scores
First, in order to add the average author's scores we need to calculate the average author's scores and render those scores in the review report UI.
Overview on how to calculate the average author's scores for each team.
The questions table has all the questions. For our use case, we will be considering only the feedback questions. Answers and scores to these questions will be saved in the Answers table with question id being the foreign key. Now, the tricky part is to know whether the answer is feedback given by an author or the review given by the reviewer. We will make use of the Response Map table to find this. Response id column in the Answers table will map to the id column of Response table with response id being the foreign key in the Answers table. The map_id column in the Response table will map to the id column in the Response Map table. The type column in the Response Map table will give us the info on whether a particular answer belongs to feedback or not. If the type of a particular answer is FeedbackResponseMap then that is feedback given by the author. For each team, the average of all authors' feedback given will be calculated which can be obtained from the Answers table.
Table Structures
Design pattern used.
1. Builder Pattern: Since the review report UI needs to render a lot of variables like reviews_hash, average and ranges hash, and author feedback scores, the underlying algorithms are complex. In order to compute each variable, it uses the builder pattern which calculates the variables one by one in their respective model classes.
Files edited as part of this change.
1. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/models/feedback_score_calc.rb - to add calculation related to average author's feedback.
2. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/helpers/report_formatter_helper.rb - to store the calculated average author's feedback as a class variable in order to access it in the corresponding HTML file. The class variable is assigned in the review_response_map method since this method will be called when a user opens the review report page.
3. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/views/reports/_review_report.html.erb - This is the actual HTML file where the review report is rendered. A new column and a toggle button to view/hide the author's feedback are added to this HTML file.
4. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/views/reports/_author_scores_avg.html.erb - this a new html.erb file. This is used to render average author feedback scores of each team.
Control Flow
2. Author's summary link
Each row in the Reviews Done column a new link called author's summary will be displayed which will redirect to the author's summary page.
1. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/views/reports/_review_report.html.erb - This is the actual HTML file where the review report is rendered. A new column and a toggle button to view/hide the author's feedback are added to this HTML file.
3. Author's summary page
A new HTML page view_author_scores_popup.html.erb will be created to render the author feedbacks. Since this HTML page will be a bit similar to the summary page, we will make use of view_review_report_popup.html.erb in order to follow DRY principle.
1. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/controllers/popup_controller.rb - a new controller method is added which helps us in redirecting to the new author's feedback page where all the comments of authors for a reviewer will be displayed.
2. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/models/review_response_map.rb - logic to store the response ids of all author feedbacks is added in this file. All the response ids will be stored into a hash with review rounds as keys. These response ids are used to retrieve author comments towards a reviewer when rendering author's feedback page.
3. https://github.com/VarunVeginati/expertiza/blob/author-feedback/app/views/popup/view_feedback_scores_popup.html.erb - A new HTML file called view_feedback_scores_popup.html.erb is created in order to display the new author's feedback page.
Test Plan
Manual Testing
Through manual UI testing, we aim to identify if all the features that add are working as intended.
Steps for manual testing
1. Login as an instructor.
2. Click on Manage->Assignments.
3. Click on the view reports icon.
4. Select Review report and click on view.
5. By default the review report will not have the author's average scores.
6. When the toggle button is clicked, a new column should be added without the page being refreshed.
7. A new author's summary link should be available in each row under the Reviews Done column.
8. When clicked on the author's summary link, it should redirect to a new author's summary page where the author's scores and comments should be rendered teamwise.
Rspec testing
1. https://github.com/VarunVeginati/expertiza/blob/author-feedback/spec/models/feedback_score_calc_spec.rb - An Rspec test is added for feedback_score_calc_spec.rb to test the calculation of the feedback author score hash.
Command to run the test case: bundle exec rspec spec/models/feedback_score_calc_spec.rb
2. https://github.com/VarunVeginati/expertiza/blob/author-feedback/spec/controllers/popup_controller_spec.rb#L48 - An RSpec test case is added for popup_controller.rb to test whether the author feedback page is rendering with 200 status or not.
Command to run the test case: bundle exec rspec spec/controllers/popup_controller_spec.rb
3. https://github.com/VarunVeginati/expertiza/blob/author-feedback/spec/models/review_response_map_spec.rb#L476 - An RSpec test case is added for review_response_map_spec.rb to test the generation of feedback_final_versions hash.
Command to run the test case: bundle exec rspec spec/models/review_response_map_spec.rb
Pull Request
https://github.com/expertiza/expertiza/pull/2134
Project Walk-through
https://www.youtube.com/watch?v=3ABUSDCBirc
Team
Team Members
Varun Kumar Veginati [vvegina@ncsu.edu]
Laxmi Aishwarya Thela [lthela@ncsu.edu]
Joshua Myers [jamyers3@ncsu.edu]
Lalitha Mulakaluri [lmmulaka@ncsu.edu]
Mentor Kai Xiao [yxiao28@ncsu.edu]