CSC/ECE 517 Fall 2016/E1705. Tracking the time students look at the others' submissions

Introduction

Purpose

Expertiza provides the functionality of peer reviews. The best advantage of peer review for the students is that it allows students to learn from each other work.

1. It provides a dual benefit for both the reviewer and the writer as they are introduced to a different set of eyes on the code
2. It might expose edge cases not previously thought of
3. It helps instructors review a project faster
4. It emphasizes the importance of writing in such a way that it's easy for a third person to read and understand

In order to assess and analyze quality of reviews, we want to be able to track the time that students spend observing the other's work. Unfortunately, most peer assessment systems do not manage the content of students’ submission within the systems. They usually allow the authors submit external links to the submission, which makes it difficult for the system to track the time that the reviewers spend on the submissions. To assess and improve review qualities, the purpose of the project is to be able to successfully track the time that each student spends on peer reviewing the work.

What needs to be done

1. Design and implement a solution that handles the use case, where students’ submissions are hosted on an external website e.g. GitHub and Youtube use cases
2. Modify the reviewer interface in expertiza to accommodate your solution e.g., change the link that opens submitted files in views/response/_submitted_files.html.erb so that when the user clicks on the link, it opens a popup window that shows the submission, record time when it opens and closes
3. Design a database schema for logging the time a reviewer spend on a submission
4. Modify Review report (views/review_mapping/_review_report.html.erb) to show the time spent on each submission

Approach and Implementation

Types of submissions

Based on each submission type, we need to handle the tracking functionalities differently. Generally, the submission links can be one of the following:

1. GitHub project/repository link , GitHub Pull requests link or Deployment link
2. Youtube videos
3. Wikipages
4. Text document/doc/pdf/ppt
5. Images - png/jpg
6. Downloadable contents

Steps to follow for all external links (Github, YouTube etc)

On clicking a submitted link available in submitted_content/submitted_files.html.erb, a new window will pop-up that will keep track of the time spent on that new window. This can be a link to any of the above-mentioned submissions.
Our solution is to embed the submissions (done on an external website) within the peer assessment system itself using e.g., IFrame or Embed tag. We shall include two new javascript files - TimeMe.js and IfVisible.js to keep track of open submission links and time spent viewing them.
The following are the steps that will be followed to keep track of the time:

1. Start the time when a user clicks on the submitted link (a new window pops-up)
2. We'll pause the time in the following cases:
   1. User idle for more than 30sec
   2. User switched tabs
   3. User minimized the window
3. Resume time when:
   1. User switches back to this window
   2. User performs some activity after being idle
4. Time is saved into the database when the user saves, submits review or closes the popped-up window
   1. If the corresponding entry already exists, the value of the time column is updated otherwise it is created
      
      

Behaviour Diagram :

Instructor can now view the times taken for reviewers by reviewer in the Review report summary. This report is rendered by partial review_mapping/_review_report.html.erb. We shall modify this view to record the times spent by the reviewer per review round for each assignment. For example - 15.32/12.67 indicates that reviewer spent 15.32 minutes on round 1 review and 12.67 minutes on round 2 review.

Modified View :

Edge cases

Edge cases that need to be handled :

1. What happens when the user refreshes the window : It will set the timer back to 0 and the time spent on that page will be reset
2. What happens when the user downloads the submission: If the user downloads any solution and then opens it in his/her local machine, then the time spent on that can't be tracked

Database Design

The following table review_submission_timing will be created to implement a solution for this project:

Files to be modified/added

1. submitted_content/submitted_files.html.erb
2. review_mapping/_review_report.html.erb
3. review_mapping_helper.rb (method- get_data_for_review_report)
4. Additionally, we will create a new model class corresponding to new database table
5. TimeMe.js and IfVisible.js for handling timing counter

Test Plan

To test if the time spent by the reviewer on the submitted links shows up in the response_report in Instructor View.
Steps:

1. Login as a reviewer, for example (student5930). Click on the submitted links to start reviewing. Once review is done, the time spent is recorded in the background and saved in our database.
2. Login as an instructor. Click on assignments tab. Click on 'View Review Report' under actions for the particular assignment. In the view rendered we expect the new content of time spent by the reviewers to be visible.