CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

Introduction

The expertiza project takes advantage of peer-review among students to allow them to learn from each other. To track the review time that each student spend on each submitted resources is meaningful to instructors to study and improve the teach experience. However, the time tracking of specific content on these resources is not perfect in the expertiza system. The previous team working on E1705 project has solved part of such problems by tracking the active time of windows opened from the submitted links. But for the submitted files which could be downloaded for local review, there is only a rough approximation solution yet. In this project, our team is going to solve this problem as well as improving previous time tracking methods using mature third-party APIs to record link review time more accurately. To accomplish this goal, here are the general solutions we designed and implemented in this project:

• Designed a database schema for logging the time a reviewer spend on a submission;
• Designed and implemented dynamic partial components using Javascript and Ruby to open students’ submissions in new windows to view online;
• Used DOM to control the windows holding review submission files or links, and record the open and close time;
• Modify Review report (views/review_mapping/_review_report.html.erb) to show the total and detailed time spent on each review submissions with visualization.

Problem Description and Solution

Our team is going to solve the submission review time tracking by attempting to open the downloaded files inside popup windows of the web browser in order to record the time on the client side.

Type of submitted files

For most of our tasks, there are two types of submitted files.

1. One kind is online links, such as github repo or pull request link, youtube video link, expertiza wiki page link, etc

2. The other one includes files such as pdf, txt, doc, images, etc and other download contents. (instructor can include more document types in /views/submitted_content/_submitted_files.html.erb)

Design pattern

Strategy In this project, for each review page, we need to deal with different type of links. For online links, such as github repo, youtube link, etc, we use javascript to open a new window detect time when the window is closed. For submitted files, such as txt, jpg, we will open it with views/response/_submitted_files.html.erb. For other files that need to be download and open locally, we can only make approximation for review time.

Database schema design

For this project, we need to design a database table to store all the time that we need to record for each review submission event. We would like to name the table as submission_viewing_events. For the table, we would use map_id in table response, which uses response_map_id as the primary information. Then we need to record reviewer_id and reviewee_id, source_link for submitted links or file names, as well as time start_at and end_at, then store them in the table.

Time tracking mechanism

For each review, when the reviewer click on one link of the submitted files, we create an entry to the table review_time, we will write a function to record current time to the database.

For how to determine time when the reviewer complete with the opened link, there are different cases. If the reviewer click on save or submit button, close the page, or logout, we need to write to all open entry (without end_at) respect to this review and save current time to end_at. But for how to decide when the review close the opened link is a little complicated. For github and youtube links and txt, pdf, doc, image files, we will open submitted files in views/response/_submitted_files.html.erb so that when the user click on the link, it opens a popup window that shows the submission, record time when it opens and closes. This is done by controlling DOM using jQuery javascript library, and then send back the time records through Ajax. For the other files that need to be downloaded, we can only make an approximation by record the time when the reviewer click on the link until the time he/she enter the reviews.

For each task, reviewer might open each link several times, might also save the review process and restart it some times later, so we will have multiple records for each link. Every time the student click on the link, a new record is created and then the time in all records will be summed up to total time for every link and displayed on instructor page.

View modification

After we record start_at and end_at time for each piece of review, we need to display time that each reviewer spent on each link. Because the reviewer may save the review and start again in the future, so for each link there might be several entries, we need to sum them together. We modifed code in review_mapping/_review_report.html.erb first to modify the view. This framework follows well the "open to extension and close to modification" principle, which makes further changes more feasible by adding additional view components without affecting current functions.

Edge cases

1. When user click on save or submit the review without closing opened windows, the system will search all records of response time for the reviewer and reviewee where end time is NULL, and update it with current time.

2. If the reviewer opens the reference links or download files, but leave them open and turn to focus other unrelated stuff, the review page will send a popup alert dialog to confirm if the review is still in review mode. If not, it will close these windows and save the time as approximation of end time. Otherwise, it will continue the review process.

3. In some cases, reviewer might shut down the computer, close the browser, log out without saving, etc., we set up periodically saving reviewing events automatically such that we will lose as less information as possible. Currently the period is 30 seconds and this can be changed. For the cases described, shut down computer etc., there will be records written in data base with end time NULL, next time when the reviewer restart the review, the system will just drop all the records with NULL end time and start a new query. Also if reviewer does not interact with the review page for a long time, it will pause the timer and commit all existing related time record in database. The time limit is now 2 minutes. Both the period and non-interaction time can be adjusted by instructors in code file /views/response/response.html.erb

Test Plan

UI and Manual test

We will test the following cases during our development

1. When we work the review, if we click on GitHub link, GitHub pull request link, check whether the system record review time correctly.

2. When we work the review, if we click on doc, pdf, image link, check whether system open submitted files in views/response/_submitted_files.html.erb and recore the review time correctly.

3. Check if the reviewer save the process and start again with click on same clink, check whether the system will record all the entry, and on the view page take all entries for one link into account.

Rspec Test

Test of new model and function will be written and tested during project development:

• Test the success of a review time record given valid start and end time;

describe ResponseTimesController do

 describe '#action_allowed?' do
   it "should return true" do
     expect(true).to be_truthy
   end
 end

• Test whether the start time is updated to the end time last saved in the database when the user resumes a review;

 describe '#response_start_time' do
   context 'when the link is opened and timed' do
     it 'should update time record with start time as end time' do
       response_time_records=double('ResponseTime')
       allow(ResponseTime).to receive(:where).with([:map_id,:round,:link]).and_return(response_time_records)
       dummy = double('BasicObject')
       allow(ResponseTime).to receive(:end_at).and_return(dummy)
       allow(dummy).to receive(:nil?).and_return(true)
       allow(ResponseTime).to receive(:update_attributes).with(:end_at,:start_at).and_return(response_time_records)
       expect(response.body).to be_blank
     end
   end
 end

• Test whether the end time is updated as the current time when the review is saved to the database.

describe '#response_end_time' do
   context 'when response does not have a end time' do
     it 'should update time record with end time as current time' do
       response_time_records=double('ResponseTime')
       allow(ResponseTime).to receive(:where).with([:map_id,:round,:link]).and_return(response_time_records)
       dummy = double('BasicObject')
       allow(ResponseTime).to receive(:end_at).and_return(dummy)
       allow(dummy).to receive(:nil?).and_return(true)
       allow(ResponseTime).to receive(:update_attributes).with(:end_at,Time.now.to_date).and_return(response_time_records)
       expect(response.body).to be_blank
     end
   end
 end

Related Files

submission_viewing_events.rb
submission_viewing_events_controller.rb
responses_controller.rb
helper/review_mapping_helper.rb
helpers/submitted_content_helper.rb
response/response.html.erb
review_mapping/_review_report.html.erb
review_mapping/_review_submissions_time_spent.html.erb
view/popup/reviewer_details_popup.html.erb
view/submitted_content/_hyperlink.html.erb
view/submitted_content/_submitted_files.html.erb
db/migrate/20171117190721_create_response_times.rb
config/routes.rb

Future Work

In this project, we accomplished the requirement defined in the project instructions. However, due to the complexity of the Expertiza system and limited time period, there are still several points to improve in the future:

1. Improve the accuracy of review time record. To record the exact time that the user spends on each link or file, it is more accurate to track the active time of each new window or tab of these links or online files. We attempted this design using 'window.focus()', but it does not work once the user moved away from the review page and turned to other applications. So either tweaking the web setting or using browser APIs may solve this problem.

2. Notify the user when no response on the review page. Occasionally the user may open the review page to do the review, but is interrupted to focus on other applications. In this case, we stopped the time recording but try to alert the user that if he wants to continue the review. However, this notice cannot show in time if the user is not focusing on that review page. So a global alert is necessary to save the time and provide notice on time.

Project Links

1. Project Github repo

2. Project Video Demo

Team

Shijie Li
Wei Sun
Aishwarya Sundararajan
Darshan Balakrishna Bhandari