CSC/ECE 517 Fall 2017/E1799 Improve self-review Link peer review & self-review to derive grades
Introduction
Purpose
The purpose of the project is to improve self-review, link self review and peer review, to device a strategy to come up with the composite score.
Problem Definition
- 1. Implementation of the concept to see self-review scores juxtaposed with peer-review scores. Designing a way to show them in the regular “View Scores” page and the alternate (heat-map) view
- 2. Implementation of the methodology to combine self-review and peer-review scores to derive a composite score
- 3. Checking if the author has submitted the self-evaluation(s) before seeing the results of their peer evaluations(already implemented).
Overall Description
It should be possible to see self-review scores juxtaposed with peer-review scores. The way to show them in the regular “View Scores” page and the alternate (heat-map) view should be designed. They should be shown amidst the other reviews, but in a way that highlights them as being a different kind of review.
Then implementation of the way to combine self-review and peer-review scores to derive a composite score should be done. The actual formula is not as important as it is to encapsulate the formula in a method that can easily be modified if we want to change the formula. But the basic idea is that the authors get more points as their self-reviews get closer to the scores given by the peer reviewers. So the function should take as inputs the scores given by peers to a particular rubric criterion, and the score given by the author(s). The result of the formula should be displayed in a conspicuous page on the score view.
Of course, there would be no challenge in giving the same self-review scores as the peer reviewers gave if the authors could see peer-review scores before they submitted their self-reviews. So the author(s) should be required to submit their self-evaluation(s) before seeing the results of their peer evaluations.
Application Flow
The instructor has to login and create an assignment by checking allow self review option. The instructor has to add the topic and assign students to the topic. Now, the student has to login and submit the assignment. The self review link is visible in the page. The student should give the self review. After giving the self review, the student should be able to see the peer review, self review and the composite score.
Use case Diagram
Actors:
- Instructor: This actor is responsible for creating assignments and adding students to the assignment.
- Student: This actor is responsible for submitting, self-reviewing and viewing the scores.
Database:
- The database where all the data of Expertiza is getting stored.
All the other use cases are implemented except “View Scores with self-review column and composite score”
Use Case: View score with self-review column and composite score
Pre Conditions:
1.The Student should submit the assignment and self-review.
2.The other students should submit the reviews of the work submitted.
Primary Sequence:
1.The student should login.
2.The student should browse and upload the assignment.
3.The student should submit the assignment.
4.The student should submit the self-review.
5.The student should choose your scores to view the score.
6.The student should be able to see the peer-review score with the self-review scores and composite score.
Post Conditions:
1.The self-review and composite score will be visible on the page.
Database Design
The database used here is Expertiza.
1.The self-review data is getting stored in the existing table. This data is fetched from the table and shown in the view with the other peer-review columns.
2.The peer-review data and self-review data is fetched from the existing table and passed to the method. The method computes the composite score and is shown in the view.
Method to calculate the composite score
Design decision
We considered Mean, Median and Mode to calculate the composite score. But Median and Mode implementation has following problems.
Median - The reviews are not too skewed to use median, also median is less inefficient as data sorting has to be done to find median
Mode - This causes trouble if there are multiple modes(values with same frequency) then data would be improper.
The above problems can be solved by using mean.
The above data is taken from the actual reviews done for the third OSS project.
1.The average has been calculated for the reviews done by the other teams.
2.The weighted average is calculated by dividing average by maximum score.
3.The weighted self-review is calculated by dividing self-review score by maximum score.
4.Difference value is calculated by subtracting weighted average and weighted self review.
5.The sum of differences of all the criteria is calculated which gives the deviation from the total of review scores(Difference score).
6.The difference between the number of questions and the Difference score gives us the composite score. And divide the composite score by the number of questions and multiply by 100 to get the final composite score.
| Range | Self Review Quality |
| <30 | Improper assessment |
| 30 to 60 | Moderate |
| 60 to 80 | Can do better |
| 80 to 100 | Perfect |
Design Pattern
In our project, we have not implemented any new design pattern because the implementation does not involve any design level changes. We have used the existing classes to incorporate the self review, hence there are no class level changes.
However, Expertiza uses the below design concept.
1.MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).
2.Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.
Code Modifications
In app/views/grades/view_team.html.erb, the html code to display self review column has been added beside the peer review scores. The final self review score has been displayed below the table.
In app/models/vm_question_response.rb, the method to calculate the composite score has been added.
In app/models/response_map.rb, the self review logic in the condition statement mainly checking for the type of response map has been included.
In app/models/vm_question_response_row.rb, the method to calculate the weighted average, which is used to calculate the overall composite score has been included.
Implementation
After logging in as a student, the user should select the assignment. The user can see the attributes related to the assignment. The user selects "your scores" option to view the below page. The last column has the self review scores. The composite score(self review score) is calculated and is displayed at the bottom of the table.
After logging in as a student, the user should select the assignment. The user can see the attributes related to the assignment. The user selects "alternate view" option to view the below page The composite score(self review score) is displayed beside the final review score.
After logging in as a instrcutor, the user should select the assignment. The user can see the attributes related to the assignment. The user selects "your scores" option to view the below page. The same page "your scores" which is used for student view has been used in the instructor view as well. The composite score(self review score) is displayed beside the final review score.
Test Cases
| Test case ID | Test Objective | PreConditions | Steps | Test Data | Expected Result | Post Conditions |
| E1799_Test_Case_1 | Successfully displaying self review scores in the heat map view | 1. The student should do the self review.
2. The scores of the self review should be present in the database. |
1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it. 3. Choose the self review option and complete the self review. 4. Go to view scores option. |
Submit the relevant assignment before doing the self review. | The self review column is displayed together with the other peer reviews in the end. | The student is able to see the self review column. |
| E1799_Test_Case_2 | Successfully calculating the proper composite score. | 1. The student should do the self review.
2. The score of the self review should be present in the database. 3. The function is implemented to calculate the composite score using average of the peer reviews and the self review. |
1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it. 3. Choose the self review option and complete the self review. 4. Go to view scores option. |
Submit the relevant assignment before doing the self review. | The composite score is returned by the method. | The method is able to return the expected composite score. |
| E1799_Test_Case_3 | Successfully displaying of composite score in the heat map view. | 1. The student should do the self review.
2. The scores of the self review should be present in the database. |
1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it. 3. Choose the self review option and complete the self review. 4. Go to view scores option. |
Submit the relevant assignment before doing the self review. | The composite score is displayed in the heat map view. | The student is able to see the composite score. |
| E1799_Test_Case_4 | Successfully displaying of composite score in alternate view. | 1. The student should do the self review.
2. The scores of the self review should be present in the database. |
1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it. 3. Choose the self review option and complete the self review. 4. Go to alternate scores option. |
Submit the relevant assignment before doing the self review. | The composite score is displayed in the alternate scores view. | The student is able to see the composite score. |
| E1799_Test_Case_5 | Successfully displaying of composite score in the heat map view of an instructor. | 1. The student should do the self review.
2. The scores of the self review should be present in the database. |
1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it. 3. Choose the self review option and complete the self review. 4. Login as an instructor. 5. Choose manage assignments. 6. Under assignments, view the assignment. 7. Under the corresponding assignment, go to view submissions. 8. Then select assign grade. |
Submit the relevant assignment before doing the self review and login as an instructor. | The composite score is displayed in the heat map view of an instructor. | The instructor is able to see the composite score. |
Test Results
The automation execution video is uploaded at this link
References
http://wiki.expertiza.ncsu.edu/index.php/Main_Page
Pull Request Link
https://github.com/expertiza/expertiza/pull/1094