CSC/ECE 517 Fall 2018 E1864: Issues related to Reviewing
Project Introduction & Motivation
Expertiza is an open source web-based peer review system developed and maintained by students and faculty members at North Carolina State University. It has been developed as Open Source Software using Ruby on Rails framework. Expertiza allows students to form teams, review each other's work such as assignments and projects. It provides the role based access so that instructor can impose restriction what students can view/edit.
The primary motivation behind this project is to give the students to review the projects of their peers effortlessly and fix all the issues of the previous implementation.
Description of Project
Expertiza Project : E1864 ‘Issues Related to Reviewing' is intended to fix five known issues in the current Expertiza Version related to reviewing student work. Issues are centered around peer reviewing and assignment feature calibration. The motivation behind this project is to identify and resolve each bug in an attempt to improve the current system version. The project has been divided into required fixes. Four of the five problems are presented in Figure 1 - Application Flow Diagram below. This image is a general representation describing when each of the issues is encountered during a user session. Problem 5 (Issue # 1142) is not present in this figure as the problem is simply associated with the coding of a view file and not directly connected to the flow of students and reviews. Additional discussion on proposed approaches to meet the requirements are explained in subsequent sections.
- Figure 1 - Application Flow Diagram
- -----------------------------------------------
UML diagram for reviw
Actors:
- Instructor - This actor is responsible for creating assignments and adding students to the assignment.
- Student - This actor is responsible for submitting, reviewing other students work.
Database:
- The database where all the data of Expertiza is getting stored.
Problem Statements & Descriptions
For this project, we have been asked to fix the follow bugs that have been found in the current Expertiza system :
Problem 1 : Issue #1093
- When a student requests a project to review, duplicate reviews appear in the list of available projects.
- Example : The project name for which the review was performed earlier should not appear for the next list of reviews when a student requests a review to perform. This could lead to students reviewing the same project more than once and the requirement is to review different projects each time.
Steps to reproduce the Issue #1093: Figure 1 gives the holistic view on stages in which we see the issues. In this part, we will discuss the exact steps that need to be followed to reproduce the bugs.
- Login as Instructor Create new assignment and associate to the course as shown in Figure 4
- Add a participant to the assignment. there are two ways to do this in bulk first way is importing all the course participants to assignment . second approach is creating a file containing student details and import them to expertiza
- Create topics and teams in expertiza either entering them one by one or from the file
- In actual implementation topic association to the teams involves bidding process but for ease of use, we let instructor assign the topic (Figure 5). set min and max number of reviews allowed as shown in
- Set the rubrics and number of review rounds (min one round is needed) (Figure 6)
- Review assignment should be automatic not instructor controlled(scope of the stated issue is here) as shown in Figure 7
- Set the deadlines for submission and review (shown in Figure 8)
- Login as a student and submit work(this needs to be done for each team), request for review
- If the issue persist system assign the same topic to review multiple times(as shown in Figure2).difficulty with this problem is that its reproducibility factor which is very low.
Problem 2 : Issue # 1097
- When a project author resubmits their assignment, the reviewers should be able to update, not edit, their reviews. This should hold even if those reviews have already been submitted. The current system does not allow reviewers to update their reviews.
- Example : When the reviewer submits a review for a project, the reviewer cannot edit/ update again. Before submitting it says "Once submitted, the review cannot be changed again".
Real life example : When a reviewer submits the review 1 a few minutes after the deadline, the rubric for review 2 occurs. So the review 1 actually becomes review 2 and the reviewer was not able to update this review. In situations like these, update the submitted review becomes useful.
Steps to reproduce the issue:
- Initial steps to reproduce this problem is same as the issue #1093
- Change the deadlines for the review to a few days after the present date and check if you are able to update the review. The picture shows deadline to submit is after 14 days. But the reviews are only able to be viewed.
Problem 3 : Issue # 1029
- When a reviewer tries to review a project, sometimes he/she is shown that the project has not yet been submitted when in fact it has.
- Example : An author of a project has submitted his/her assignment to Expertiza. When a reviewer tries to review the work, sometimes it happens that the system shows the reviewer that the work has not been submitted yet. The reviewer should have been shown the work of the author to review.
Steps to reproduce the Issue #1029:
- steps to reproduce this issue is similar to issue #1093, only the difference is once reviewer requests the assignment to review system throws a message saying there is no work has been submitted even though there is actually submitted work. This issue is not always reproducible
Problem 4 : Issue # 972
- When updating the number of review rounds for an existing project, and error is generated and this action fails.
- Example : An instructor attempts to edit an existing assignment and change the number of review rounds. The reassignment causes an error in the system and this prevents the project update to occur.
Steps to reproduce the Issue #972 :
- We will choose one of the existing assignments in expertiza and try to increase the number of review rounds from "x" to "y"(x<y). when we try to save the assignment we get the error message as shown in the figure 12 below also number of reviews reverts back to "x".
Problem 5 : Issue # 1142
- “Summary Result Assignment” page shows raw HTML when it should displayed as processed HTML.
- Example : As seen in Figure 16 below, the result of hovering over a review result displays the message in raw HTML. The intended result is for review comment to be rendered properly while displaying.
- -It should be that this issue can be reproduced by going to any student assignment result and attempt to produce a quick view of the review comment by hovering over the desired linked question score.
Files to be Modified
- app/models/review_assignment.rb
- app/controllers/assignments_controller.rb
- app/views/view_team.html.erb
Solution Design
Problem 1 : Issue # 1093
- If the issue reproducible consistantly we need to fix reject_previously_reviewed_submissions(contributor_set, reviewer) method in review_assignment.rb model file. This method is responsible for the seperating the current review topic and previously reviewed topics. solution flow diagram is shown below
Problem 2 : Issue # 1097
- A possible solution is to let the reviewers update the review until the deadline and possibly remove the message "Once submitted, it cannot be changed again". We can let the reviewers update their reviews multiple times before the deadline.
Problem 3 : Issue # 1029
- Solution to this issue involves identifying students work as correctly submitted or not submitted so that system will always know that submission status this identification should be available to the method which assigns topics to the reviewer.
Problem 4 : Issue # 972
- To solve this problem we must first identify the necessary actions for which the review round updates are to be executed. From there, we can step through the calls to determine which is not executing as intended. After testing, there is a suspicion that the error is occurring due to the response by, or lack there of, the 'set' button on the edit page beside the rounds value box. The response of this action button determines the outcome of the conditional statement that must evaluate to true to properly update the number of review rounds. Additionally, the 'set' button seems to be inactive upon the first render of the edit page but active on subsequent visits. This update action can be found in the update_feedback_assignment_form_attributes function of assignments_controller.rb
Problem 5 : Issue # 1142
- In this case, we need to identify the mistakes in HTML file and then correct them. final expected outcome is to have processed HTML file.
- - We believe that this issue stems from one of two places:
- (1) The reviewer content is being captured and stored into the database as a string still containing the html formatting and then being populated into the view as a string containing those tags.
- (2) There is a tag missing from view_team.html.erb where this hovered content is being requested. There are eight instances where hovered content is referenced. More testing review of the view file will lend to narrowing down the actual line where the problem resides.
Testing Plan
Testing Plan will be implemented upon the discovery of fixed bugs described in the project document as a means of verifying the fix.
To ensure correctness of bug fixes following test plan was implemented:
- Obtain the appropriate testing environment via a provided virtual box image and RSPEC testing framework.
- Acquire a deeper understanding of the necessary models and views, their functionalities and dependencies.
- Create factories and doubles to assist in the testing of these models and views methods.
- Mock message passing and expected outcomes.
- Apply generated helper objects and mocks to achieve high test coverage of methods within the models and views that were updated.