CSC/ECE 517 Spring 2019 E1926 Improve self-review

E1926. Improve self-review, Link peer review & self-review to derive grades

This page provides a description of the Expertiza based Semester project.

About Expertiza

Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

Application Flow

The above image shows how instructor will allow users to give a self review on the work they have done. Instructor have to go to the review strategy tab while assignment creation to enable the self review option.

User then can see the self review option on the assignment to do tasks to give a self-review.

In the view scores page a new collumn showing the self review scores will be displayed.

Use case

Actors:

   Instructor: This actor is responsible for creating assignments and adding students to the assignment. 
   Student: This actor is responsible for submitting, self-reviewing and viewing the scores. 

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Preconditions:

  1.The Instructor creates Assignments and checks the Allow Self Review check box in the Assignment
  2.The Student should submit the assignment and self-review.
  3.The other students should submit the reviews of the work submitted.

Student Sequence:

  The student should login.
  The student should browse and upload the assignment.
  The student should submit the assignment.
  The student should submit the self-review.
  The student should choose your scores to view the score.
  The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

  The self-review and composite score will be visible on the page.

Formula Proposed

The composite score is calculated using both self-review score and peer-review score (both are graded over 5 points for each criteria). The composite score is calculated as follows:

  composite score = [10 - abs(self-review score - peer review score)] * peer review score / 10

By using this formula for calculating composite score we are discouraging students from either exaggerating or underrating their performance. The composite score model assigns more weight to the peer review score since the peer review score being used is the average of all peer review scores assigned and hence tends to be more appropriate than a single self-review.

Also this composite score formula gives the score on a scale which is identical to the scale of the self and peer review score. Thus this formula is not dependent on the scale of the self-review and peer-review score as it will also return the composite score on the same scale.

For example: If someone gets an average peer review score of 3.5/5 and he gives 3.5/5 (self-review score) while reviewing his project, then he will get a composite score of 3.5/5. And instead if the student reviews his project for 2/5 then the student will get a composite score of 2.975/5.

Implementation

As stated above the ability to do self-reviews has already been implemented. This can be seen from the image shown below. The instructor has this ability to enable self-reviews .

In our project, we were successful in displaying both the self-review score and composite review score on the view scores page. Shown below is an example of the output in the student profile. As we can see the self-review icon has a blue background to indicate that it is different from other peer reviews.

The alternate view was also modified to accommodate a pictorial representation of the self-review scores. The modified alternate view scores page is shown below.

The same was achieved in the instructor profile too. In the current implementation of expertiza when self-review was not assigned, the instructor would view the scores of the first student in each team (it wouldn't matter much because without self-review all students in a team would have the same scores). But this would be a problem when self-review was enabled so we decided to display all the self-review scores of all students in the team for instructor view. As it can be seen below self-reviews of all students in the team (distinguished by their usernames) are indicated by blue background to differentiate them from peer review scores.