CSC/ECE 517 Fall 2016/E1705. Tracking the time students look at the others' submissions

Introduction

Purpose

Expertiza provides the functionality of peer reviews. The best advantage of peer review for the students is that it allows students to learn from each others' work.

1. It provides a dual benefit for both the reviewer and the writer as they are introduced to a different set of eyes on the code
2. It might expose edge cases not previously thought of
3. It helps instructors review a project faster
4. It emphasizes the importance of writing in such a way that it's easy for a third person to read and understand

In order to understand the correlation between the time spent in reviewing and the knowledge gained through reviewing submissions, we need to track the time that each student spends on peer reviewing the work. The knowledge gained can be measured by assignment score and reviewing score of the reviewer. This data can be further be utilized for research studies related to learning through peer reviews.

What needs to be done

1. Design and implement a solution that handles the use case, where students’ submissions are hosted on an external website e.g. GitHub and Youtube use cases
2. Design a database schema for logging the time a reviewer spends on each submission link
3. Modify Review report (views/review_mapping/_review_report.html.erb) to display the response time summary report for each reviewer.

Approach and Implementation

Types of submissions

Generally, the submission links can be one of the following:

1. GitHub project/repository link , GitHub Pull requests link or Deployment link
2. Youtube videos
3. Wikipages
4. Text document/doc/pdf/ppt
5. Images - png/jpg
6. Downloadable contents

Database Design

The task is to know how much time a reviewer spent reviewing other teams.

1. We earlier thought of having response_id as one of the fields which could help us identify for which response we are recording reviewing time. But, entry in the responses table is created only once the response is either saved or submitted. Since we want our response_times entry to be created before that, we did not have any response_id corresponding to the response.
   
2. We primarily needed reviewer and reviewee specific data. So, another parameter to help us identify this primary information was response_map_id in the response_maps table. Once the review is started , we have the response_map_id from which we can later derive both reviewer_id and reviewee_id.There can be separate links uploaded per round, hence round is also an important parameter to be recorded. We need to record start and end time per link.
   

The following table response_times was created to implement a solution for this project:

How reviewing times get recorded in the database

Start and end time of each link viewed in one session needs to be recorded.

The following are the steps that will be followed to keep track of the time:

1. Log the start time when a user clicks on a submitted link (a new entry in the response_times table gets created) during new review or while editing his saved review. An AJAX call is made from the view to ResponseTime controller action record_start_time to save start time details to database.
2. End time of all open links (where end time has not been recorded yet) for that session get recorded in the cases mentioned below. In this, an AJAX call is made from the view to ResponseTime controller action record_end_time to save end time details to database. All these cases are handled with the help of javascript onbeforeunload property
   1. User logged out
   2. User closed the window
   3. User clicked on 'save' or 'submit' button
   4. User switched to some other view other than the page where he is currently writing the review in the same tab. Note - the user is free to switch between tabs. The end time will not get affected in this case
3. Next time the reviewer opens the links again to review:
   1. New entry gets created in the response_times table and start time gets recorded for this new session
   2. The above procedure for end time remains the same
4. In case there is no user activity on the expertiza reviewing page for more than 5 minutes, there will be a pop up asking the user if he is still reviewing the submission.
5. As soon as the pop up is shown, the end times of all open links get updated to this time. The user has two options to cancel or to okay. If the user clicks on 'okay', then the above links for which we just updated the end time, need to be added as new entries again in this new session with start time as the current time.
6. This way we have individual entries for each link corresponding to each session with both start and end times recorded in the database.

Behaviour Diagram :

Viewing the recorded times spent

Instructor can now view the total times taken by reviewers in the Review report summary. This report is rendered by partial review_mapping/_review_report.html.erb. We shall modify this view to record the times spent by the reviewer per review round for each team reviewed for this particular assignment. For example - 15.32, 12.67 indicates that reviewer spent 15.32 minutes on round 1 review and 12.67 minutes on round 2 review as shown below.

Modified View :

Now, we can see that this report is simply an overview. If the instructor wishes to view fine-grained details, he can click on the reviewer link in the report. A popup is then rendered by popup/reviewer_details_popup.html.erb. This view shows reviewer details and details of time spent per link per round by that particular reviewer. Note - total time represents total time spent by the reviewer over all rounds over all links.
If the user clicks on "alternate view" for a particular round, a pie-chart representation of the same data for that round appears above the details table. The user can toggle this view off by clicking "hide details" and "show details".

Other Edge cases handled :

1. What happens when the user clicks on the same link twice in the same session:
   1. It may so happen that once the user clicked on a link, by mistake he/she closed it or lost the link and wishes to view it again. If the link is again clicked, a second entry gets created with the current time as start time.
   2. We account for the previous entry by updating its end time as current time.
2. What happens when the user downloads the submission:
   1. If the user downloads any submission, the start time will be recorded on click link and end time will get recorded as above or when he submits the review

Files modified

1. response_controller.rb
2. response/response.html.erb
3. submitted_content/_main.html.erb
4. submitted_content/_hyperlink.html.erb
5. response/view.html.erb
6. review_mapping/_review_report.html.erb
7. popup/reviewer_details_popup.html.erb
8. helpers/review_mapping_helper.rb
9. config/routes.rb
10. application.js

Files added

1. response_time_controller.rb
2. response_time.rb
3. review_mapping/_review_submissions_time_spent.html.erb

Testing

Manual Testing

To test if the time spent by the reviewer on the submitted links shows up in the response_report in Instructor View.

1. Login as a reviewer, for example (student5930). Click on the submitted links to start reviewing. Once review is done, the time spent is recorded in the background and saved in our database
2. Login as an instructor. Click on assignments tab. Click on 'View Review Report' under actions for the particular assignment. In the view rendered we expect the total time spent by the reviewers for each to be visible
3. Click on the reviewer in order to see detailed information of time spent by each reviewer
4. Click on alternate view for each entry in order to see link wise time spent in form of a pie chart.

RSpec Testing

To test the two new controller methods, we wrote and ran RSpec tests for it. We have two methods in the new controller - record_start_time and record_end_time.

• In the following code, we are checking that there is a successful POST request to both the methods. To record the starting and ending time of a person looking at the submission, we call 'record_start_time' and 'record_end_time' respectively. We create a stub student user and a record in response_times database to imitate the real data entry.

 
require 'rails_helper' 
include LogInHelper

RSpec.describe ResponseTimeController, type: :controller do

context 'checking that the user creates valid post request' do

  date1 = DateTime.parse("2011-05-19 10:30:14")
  date2 = DateTime.parse("2011-05-19 11:30:14")
  date3 = DateTime.parse("2011-05-19 13:30:14")

  let!(:responsetime) {ResponseTime.create(map_id: 1, link: 'hello', round: 1, start: date1)}
  let!(:first_review) { Response.create(map_id: 1, additional_comment: 'hello', round: 1) }
  let(:response_time) {ResponseTime.create(map_id: 2, link: 'hello2', round: 2, start: date2, end: nil)}

  before(:each) do
    student.save
    @user = User.find_by_name('student14')
    @role = double('role', super_admin?: false)
    stub_current_user(@user, 'student14', @role)
  end

  describe 'POST #record_start_time' do
    it 'returns with an HTTP status of 200' do

      allow(ResponseTime).to receive(:where).and_return(responsetime)
      allow(responsetime).to receive(:each).and_return(1)
      allow_any_instance_of(ResponseTime).to receive_message_chain(:new).and_return(1)
      allow(responsetime).to receive(:save).and_return(true)
      post "record_start_time", :response_time => {map_id: 1, round: 1, link: 'hello' , start: date3}
      expect(response).to have_http_status(200)
    end

  describe 'POST #record_end_time', js: true do
    it 'respond with an HTTP status of 200'  do
      allow(ResponseTime).to receive(:where).and_return(response_time)
      allow(response_time).to receive(:each).and_return(1)
      post "record_end_time", :response_time => {map_id: 2, round: 2, end: date3}, format: :json
      expect(response).to have_http_status(200)

    end
  end end end end

• The image below shows the RSpec test running successfully