CSC/ECE 517 Fall 2016/E1705. Tracking the time students look at the others' submissions

Introduction

Purpose

Expertiza provides the functionality of peer reviews. The best advantage of peer review for the students is that it allows students to learn from each other work.

1. It provides a dual benefit for both the reviewer and the writer as they are introduced to a different set of eyes on the code
2. It might expose edge cases not previously thought of
3. It helps instructors review a project faster
4. It emphasizes the importance of writing in such a way that it's easy for a third person to read and understand

In order to assess and analyze how much a student learns from others' work, we want to be able to track the time that students spend observing the the same. Unfortunately, most peer assessment systems do not manage the content of students’ submission within the systems. They usually allow the authors submit external links to the submission, which makes it difficult for the system to track the time that the reviewers spend on the submissions. To assess and improve review qualities, the purpose of the project is to be able to successfully track the time that each student spends on peer reviewing the work.

What needs to be done

1. Design and implement a solution that handles the use case, where students’ submissions are hosted on an external website e.g. GitHub and Youtube use cases
2. Design a database schema for logging the time a reviewer spends on each submission link
3. Modify Review report (views/review_mapping/_review_report.html.erb) to display the response time summary report for each reviewer.

Approach and Implementation

Types of submissions

Generally, the submission links can be one of the following:

1. GitHub project/repository link , GitHub Pull requests link or Deployment link
2. Youtube videos
3. Wikipages
4. Text document/doc/pdf/ppt
5. Images - png/jpg
6. Downloadable contents

Overview of implementation flow

Start and end time of each link viewed in one session needs to be recorded.

The following are the steps that will be followed to keep track of the time:

1. Log the start time when a user clicks on a submitted link (A new entry in the response_times table gets created
2. End time of all open links (where end time has not been recorded yet) for that session get recorded in the following cases
   1. User logged out
   2. User closed the window
   3. User clicked on 'save' or 'submit' button
   4. User switched to some other view other than the page where he is writing the review currently.

Note- the user is free to switch between tabs. End time will not get affected in this case

1. Next time the reviewer opens the links again to review:
   1. New entry gets created in the response_times table and start time gets recorded for this new session
   2. The above procedure for end time remains the same
2. In case there is no user activity on the expertiza page for more than 20 minutes, there will be a pop up asking the user if he is still reviewing the submissions. As soon as the pop up is shown, the end times of all open links get update to this time.
3. This way we have individual entries for each link corresponding to each session.

Behaviour Diagram :

Instructor can now view the times taken for reviewers by reviewer in the Review report summary. This report is rendered by partial review_mapping/_review_report.html.erb. We shall modify this view to record the times spent by the reviewer per review round for each assignment. For example - 15.32/12.67 indicates that reviewer spent 15.32 minutes on round 1 review and 12.67 minutes on round 2 review as shown below.

Modified View :

Edge cases

Edge cases that need to be handled :

1. What happens when the user clicks on the same link twice in the same session: It may so happen that once the user clicked on a link, by mistake he/she closed it or lost the link and wishes to view it again. If the link is again clicked, a second entry gets created with the current time as start time. We account for the previous entry by updating its end time as current time.
2. What happens when the user downloads the submission: If the user downloads any solution and then opens it in his/her local machine, then the time spent on that can't be tracked

Database Design

The following table review_submission_timing will be created to implement a solution for this project:

Files to be modified/added

1. student_review/list.html.erb
2. review_mapping/_review_report.html.erb
3. review_mapping_helper.rb (method- get_data_for_review_report)
4. Additionally, we will create a new model class corresponding to new database table
5. TimeMe.js and IfVisible.js for handling timing counter

Test Plan

Functional Testing using RSpec:
To test if the time spent by the reviewer on the submitted links shows up in the response_report in Instructor View.

1. Login as a reviewer, for example (student5930). Click on the submitted links to start reviewing. Once review is done, the time spent is recorded in the background and saved in our database
2. Login as an instructor. Click on assignments tab. Click on 'View Review Report' under actions for the particular assignment. In the view rendered we expect the new content of time spent by the reviewers to be visible

Manual Testing for Verification:

1. When user clicks on a submitted link, it opens up in a new window
2. To verify if the time displayed in the response_report view matches with the actual time spent by the reviewer for all reviews