Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2019 - E1989. Track the time students look at other submissions

2019-12-06T23:50:34Z

Fcdevita: /* Algorithm For Precise Timing Accuracy */

== ''' Introduction ''' ==
The [http://expertiza.ncsu.edu/ Expertiza] project takes advantage of peer-review among students to allow them to learn from each other. Tracking the time that a student spends on each submitted resources is meaningful to instructors to study and improve the teaching experience. Unfortunately, most peer assessment systems do not manage the content of students’ submission within the systems. They usually allow the authors submit external links to the submission (e.g. GitHub code / deployed application), which makes it difficult for the system to track the time that the reviewers spend on the submissions.

== ''' Problem Statement ''' ==
Knowing how much time a student spends on a review is helpful when determining the quality of the review itself. That being said Expertiza needs to be able to '''track''' and '''display''' how much time a student spends on each review.

The time spent on a review is the sum of multiple sources:

# The time spent on the Expertiza assignment review page itself
# The time spent looking at external pages linked from the review page
# The time spent looking at downloadable files submitted by other students

The purpose of this project three-fold:

# Gather the timing data from the sources above.
# Display the data on the "Review Report" page (views/review_mapping/_review_report.html.erb) to show the time the student spent per review. Given that previous attempts have been rejected due to UI issues, it is important that this data is displayed cleanly.
# Write tests that confirm that the above goals are functioning correctly.

=== Current Implementation ===
So far, Expertiza does not have any such feature. However, three teams have already worked on this in the past but their builds were not merged due to some problems.
#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2016/E1705._Tracking_the_time_students_look_at_the_others%27_submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement Wiki])
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])

== ''' Proposed Solution ''' ==
After investigating the prior attempts at this task, we have decided it would be best to begin our project by building off of the work done in project [https://github.com/expertiza/expertiza/pull/1124 E1791]. The reason for choosing to build off of this particular project is because they have already put the work into tracking time spent viewing external pages as well as time spent viewing certain types of downloadable files. In order to achieve our goals outlined in the [[#Problem Statement|Problem Statement]] section, the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*This will be done by using onFocus and onBlur mouse events to tell when a user is actively working on the page.
*To prevent the user from cheating the system by just keeping the review page open without doing work, a timeout feature will be implemented. After 5 minutes of mouse/keyboard inactivity, a popup will be displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page will stop being tracked until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links and downloadable files will need to be made more accurate.
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Changes need to be made so that when the student is working on the Expertiza assignment review page time is not tracked for the external links or downloadable files.

; The overall time spent on the review needs to be displayed in a table on the "Review Report" page.
*The table will look like the following table, but with one modification. The entries in "Time per review in mins" will be clickable. When clicked, they will display a popup that contains detailed information on where the time for the review was spent. For example, if a student spent a total of 22 minutes on the review, it will show that the student spent 5 minutes on the Expertiza review page, 10 minutes looking at external links, and 7 minutes looking at downloadable files. It will display these details in text format as well as graphically using a bar graph. The purpose of choosing this design is two-fold:
# It will not require the instructor to have to go to a different page every time they want more details on a particular review. Given that the review report page takes a substantial time to load, this is a necessity.
# It will prevent the table on the review report page from being cluttered by figures and too much data.

[[File:Review report view.png]]

=== Design pattern ===
'''Strategy:''' In this project, for each review page, we need to deal with different type of links. For online links, such as github repo, youtube link, etc, we use javascript to open a new window detect time when the window is closed. For submitted files, such as txt, jpg, we will open it with views/response/_submitted_files.html.erb. For other files that need to be download and open locally, we can only make approximation for review time.

=== ''' Flowchart ''' ===

Reviewer 
Every time a start time is logged for expertiza/link/file, a new entry will be created in the database.End time will be updated on the last entry present for the same link/file.
[[File:E1989_Student_flowchart.png | center]]
 
Instructor 
Time for individual links will summed and then displayed as bar graph.
[[File:E1989_Instructor_flowchart.png | center]]

== ''' Code Changes ''' ==
1. Created a controller to handle the start/end times of the files viewed by the user. => '''controllers/submission_viewing_events_controller.rb''' 

2. Created a table in database to log start and end time for each link/file. It's schema is described below. 
{| class="wikitable"
|+ Schema for table submission_viewing_events
|-
! Field
! Type
! Null
! Key
|-
| id
| int(11)
| NO
| Primary Key
|-
| map_id
| int(11)
| YES
|
|-
| round
| int(11)
| YES
|
|-
| link
| varchar(255)
| YES
|
|-
| start_at
| datetime
| YES
|
|-
| end_at
| datetime
| YES
|
|-
| created_at
| datetime
| NO
|
|-
| updated_at
| datetime
| NO
|
|}
3. Added view that will display with pie chart when instructor wants to see time spent on individual links/files. 
[[File:Pie chart.png]]

=== ''' Files that have been Added or Changed ''' ===
'''submission_viewing_events_controller.rb '''
The <tt>record_start_time</tt> function starts the time for links that have been visited by the user.
<pre>
def record_start_time
param_args = params[:submission_viewing_event]
# check if this link is already opened and timed
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: param_args[:map_id], round: param_args[:round], link: param_args[:link])
# if opened, end these records with current time
if submission_viewing_event_records
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
# time_record.update_attribute('end_at', start_at)
time_record.destroy
end
end
end
# create new response time record for current link
submission_viewing_event = SubmissionViewingEvent.new(submission_viewing_event_params)
submission_viewing_event.save

#if creating start time for expertiza update end times for all other links.
if param_args[:link]=='Expertiza Review'
params[:submission_viewing_event][:link] = nil
params[:submission_viewing_event][:end_at] = params[:submission_viewing_event][:start_at]
record_end_time()
end
render nothing: true
end
</pre>
 The <tt>record_end_time</tt> function records the end time for links that have been visited by the user.
<pre>
def record_end_time
data = params.require(:submission_viewing_event)
if data[:link].nil?
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], end_at: nil).where.not(link: "Expertiza Review")
else
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], link: data[:link])
end
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
time_record.update_attribute('end_at', data[:end_at])
# break
end
end
respond_to do |format|
format.json { head :no_content }
end
end
</pre>
 The <tt>mark_end_time</tt> function records the end time for links that have no end times.
<pre>
def mark_end_time
data = params.require(:submission_viewing_event)
@link_array = []
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round])
submission_viewing_event_records.each do |submissionviewingevent_entry|
if submissionviewingevent_entry.end_at.nil?
@link_array.push(submissionviewingevent_entry.link)
submissionviewingevent_entry.update_attribute('end_at', data[:end_at])
end
end
respond_to do |format|
format.json { render json: @link_array }
end
end
</pre>

reviewer_details_popup.html.erb 
_review_report.html.erb 
_review_submissions_time_spent.html.erb 
response_times.coffee 
response_times.scss 
response_controller.rb 
response_times_controller.rb 
responses_times_helper.rb 
review_mapping_helper.rb 
response_time.rb 
_submitted_files.html.erb

=== ''' Algorithm For Precise Timing Accuracy ''' ===
Previous implementations attempted to record the time users spent viewing each link, but their solution has multiple problems. The primary problem was that when multiple links were open at the same time (thus overlapping with one another) the times at which they overlap are recorded twice. To fix this, we implemented the following algorithm within \app\views\reports\_review_submissions_time_spent.html.erb:

<pre>
Consider that each link duration (the amount of time that link is being looked at) has a starting and ending edge (see diagram below). These durations are stored in an array durations_to_consider.

Iterate through every link duration in durations_to_consider:
If any other link's start and end times both fit within our start and end time:
Remove from durations_to_consider

For list of durations_to_consider, add those durations' start and end edges to edge_array.

Ensure that entries in edge_array are sorted from earliest to latest.

Create list duration_sections.

Start at the first edge in edge_array. Go until next edge. Record the time difference as a duration and add time to duration_sections.

If, after we add an edge that corresponds to the ending edge of a link duration, there are no open start edges (a start edge that belongs to a link, that does not yet have an accompanying end edge that belongs to that link):
Count the NEXT start edge as the start edge of the next duration section. This will prevent empty space caused by no link being open at that time from being counted towards the total.

Total up the time for all durations to get the total amount of time spent on links.
</pre>

'''NOTE:''' The following diagrams are simply meant to help explain the timing algorithm. They are not indicative of the real system. For example, there will never be a time when an Expertiza duration overlaps with a link duration.

The following diagram shows an example of what links could look like prior to our algorithm running:
[[File:Final_documentation_before_algorithm.PNG | center]]

After our algorithm runs, one can see that the overlapping segments are eliminated. For convenience, each duration_section is highlighted:
[[File:Final_documentation_after_algorithm.PNG | center]]

== ''' Test Plan ''' ==
=== Automated Testing Using RSpec ===
We have made changes to the controllers/submission_viewing_events_controller.rb and added test cases for the same, which have passed all the test cases. The tests can be executed <tt>rpec spec</tt> and the results for the files we have modified are shown below:

<pre>
Randomized with seed 20933
.....

Finished in 10.31 seconds (files took 7.85 seconds to load)
5 examples, 0 failures
</pre>

=== Manual UI Testing ===
We hope to not need any manual UI testing, though if it is needed, it'll look something like this:

Verifying Time is Tracked Correctly:
# Sign in as as a student
# Review an assignment
# Go through the review as normal
## Make sure you open links to github or pull requests
## Make sure you download and/or view files that are attatched
# Submit your review when you're done
# Log out as a student
# Log in as an instructor
# Navigate to the review report tab
# You should see a new column detailing the time the user spent on the review.

== ''' Our Work ''' ==
The code we created can be found below.
*[https://github.com/rohanpillai20/expertiza GitHub Repository]
*[https://github.com/expertiza/expertiza/pull/1607 Pull Request]
*[http://152.46.18.99:8080/ Deployed Link]
*[http://www.youtube.com/ YouTube Video] (To be updated.)

The project could be run locally by cloning the [https://github.com/rohanpillai20/expertiza GitHub Repository] and then running the following commands sequentially.

<pre>
bundle install
rake db:create:all
rake db:migrate
rails s
</pre>

== ''' Team Information ''' ==
#Dylan Spruill (drspruil)
#Forrest Devita (fcdevita)
#Rohan Pillai (rspillai)
#Shalin Rathi (sjrathi)

Mentor: Akanksha Mohan (amohan7)
 Professor: Dr. Edward F. Gehringer (efg)

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://relishapp.com/rspec RSpec Documentation]

CSC/ECE 517 Fall 2019 - E1989. Track the time students look at other submissions

2019-12-05T20:51:23Z

Fcdevita:

== ''' Introduction ''' ==
The [http://expertiza.ncsu.edu/ Expertiza] project takes advantage of peer-review among students to allow them to learn from each other. Tracking the time that a student spends on each submitted resources is meaningful to instructors to study and improve the teaching experience. Unfortunately, most peer assessment systems do not manage the content of students’ submission within the systems. They usually allow the authors submit external links to the submission (e.g. GitHub code / deployed application), which makes it difficult for the system to track the time that the reviewers spend on the submissions.

== ''' Problem Statement ''' ==
Knowing how much time a student spends on a review is helpful when determining the quality of the review itself. That being said Expertiza needs to be able to '''track''' and '''display''' how much time a student spends on each review.

The time spent on a review is the sum of multiple sources:

# The time spent on the Expertiza assignment review page itself
# The time spent looking at external pages linked from the review page
# The time spent looking at downloadable files submitted by other students

The purpose of this project three-fold:

# Gather the timing data from the sources above.
# Display the data on the "Review Report" page (views/review_mapping/_review_report.html.erb) to show the time the student spent per review. Given that previous attempts have been rejected due to UI issues, it is important that this data is displayed cleanly.
# Write tests that confirm that the above goals are functioning correctly.

=== Current Implementation ===
So far, Expertiza does not have any such feature. However, three teams have already worked on this in the past but their builds were not merged due to some problems.
#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2016/E1705._Tracking_the_time_students_look_at_the_others%27_submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement Wiki])
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])

== ''' Proposed Solution ''' ==
After investigating the prior attempts at this task, we have decided it would be best to begin our project by building off of the work done in project [https://github.com/expertiza/expertiza/pull/1124 E1791]. The reason for choosing to build off of this particular project is because they have already put the work into tracking time spent viewing external pages as well as time spent viewing certain types of downloadable files. In order to achieve our goals outlined in the [[#Problem Statement|Problem Statement]] section, the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*This will be done by using onFocus and onBlur mouse events to tell when a user is actively working on the page.
*To prevent the user from cheating the system by just keeping the review page open without doing work, a timeout feature will be implemented. After 5 minutes of mouse/keyboard inactivity, a popup will be displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page will stop being tracked until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links and downloadable files will need to be made more accurate.
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Changes need to be made so that when the student is working on the Expertiza assignment review page time is not tracked for the external links or downloadable files.

; The overall time spent on the review needs to be displayed in a table on the "Review Report" page.
*The table will look like the following table, but with one modification. The entries in "Time per review in mins" will be clickable. When clicked, they will display a popup that contains detailed information on where the time for the review was spent. For example, if a student spent a total of 22 minutes on the review, it will show that the student spent 5 minutes on the Expertiza review page, 10 minutes looking at external links, and 7 minutes looking at downloadable files. It will display these details in text format as well as graphically using a bar graph. The purpose of choosing this design is two-fold:
# It will not require the instructor to have to go to a different page every time they want more details on a particular review. Given that the review report page takes a substantial time to load, this is a necessity.
# It will prevent the table on the review report page from being cluttered by figures and too much data.

[[File:Review report view.png]]

=== Design pattern ===
'''Strategy:''' In this project, for each review page, we need to deal with different type of links. For online links, such as github repo, youtube link, etc, we use javascript to open a new window detect time when the window is closed. For submitted files, such as txt, jpg, we will open it with views/response/_submitted_files.html.erb. For other files that need to be download and open locally, we can only make approximation for review time.

=== ''' Flowchart ''' ===

Reviewer 
Every time a start time is logged for expertiza/link/file, a new entry will be created in the database.End time will be updated on the last entry present for the same link/file.
[[File:E1989_Student_flowchart.png | center]]
 
Instructor 
Time for individual links will summed and then displayed as bar graph.
[[File:E1989_Instructor_flowchart.png | center]]

== ''' Code Changes ''' ==
1. Created a controller to handle the start/end times of the files viewed by the user. => '''controllers/submission_viewing_events_controller.rb''' 

2. Created a table in database to log start and end time for each link/file. It's schema is described below. 
{| class="wikitable"
|-
! Field
! Type
! Null
! Key
|-
| id
| int(11)
| NO
| Primary Key
|-
| map_id
| int(11)
| YES
|
|-
| round
| int(11)
| YES
|
|-
| link
| varchar(255)
| YES
|
|-
| start_at
| datetime
| YES
|
|-
| end_at
| datetime
| YES
|
|-
| created_at
| datetime
| NO
|
|-
| updated_at
| datetime
| NO
|
|}
3. Added view that will display with pie chart when instructor wants to see time spent on individual links/files. 

=== ''' Files that have been Added or Changed ''' ===
'''submission_viewing_events_controller.rb '''
The <tt>record_start_time</tt> function starts the time for links that have been visited by the user.
<pre>
def record_start_time
param_args = params[:submission_viewing_event]
# check if this link is already opened and timed
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: param_args[:map_id], round: param_args[:round], link: param_args[:link])
# if opened, end these records with current time
if submission_viewing_event_records
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
# time_record.update_attribute('end_at', start_at)
time_record.destroy
end
end
end
# create new response time record for current link
submission_viewing_event = SubmissionViewingEvent.new(submission_viewing_event_params)
submission_viewing_event.save

#if creating start time for expertiza update end times for all other links.
if param_args[:link]=='Expertiza Review'
params[:submission_viewing_event][:link] = nil
params[:submission_viewing_event][:end_at] = params[:submission_viewing_event][:start_at]
record_end_time()
end
render nothing: true
end
</pre>
 The <tt>record_end_time</tt> function records the end time for links that have been visited by the user.
<pre>
def record_end_time
data = params.require(:submission_viewing_event)
if data[:link].nil?
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], end_at: nil).where.not(link: "Expertiza Review")
else
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], link: data[:link])
end
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
time_record.update_attribute('end_at', data[:end_at])
# break
end
end
respond_to do |format|
format.json { head :no_content }
end
end
</pre>
 The <tt>mark_end_time</tt> function records the end time for links that have no end times.
<pre>
def mark_end_time
data = params.require(:submission_viewing_event)
@link_array = []
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round])
submission_viewing_event_records.each do |submissionviewingevent_entry|
if submissionviewingevent_entry.end_at.nil?
@link_array.push(submissionviewingevent_entry.link)
submissionviewingevent_entry.update_attribute('end_at', data[:end_at])
end
end
respond_to do |format|
format.json { render json: @link_array }
end
end
</pre>

reviewer_details_popup.html.erb 
_review_report.html.erb 
_review_submissions_time_spent.html.erb 
response_times.coffee 
response_times.scss 
response_controller.rb 
response_times_controller.rb 
responses_times_helper.rb 
review_mapping_helper.rb 
response_time.rb 
_submitted_files.html.erb

=== ''' Algorithm For Precise Timing Accuracy ''' ===
Previous implementations attempted to record the time users spent viewing each link, but their solution has multiple problems. The primary problem was that when multiple links were open at the same time (thus overlapping with one another) the times at which they overlap are recorded twice. To fix this, we implemented the following algorithm within \app\views\reports\_review_submissions_time_spent.html.erb:

<pre>
Consider that each link duration (the amount of time that link is being looked at) has a starting and ending edge (see diagram below). These durations are stored in an array durations_to_consider.

Iterate through every link duration in durations_to_consider:
If any other link's start and end times both fit within our start and end time:
Remove from durations_to_consider

For list of durations_to_consider, add those durations' start and end edges to edge_array.

Ensure that entries in edge_array are sorted from earliest to latest.

Create list duration_sections.

Start at the first edge in edge_array. Go until next edge. Record the time difference as a duration and add time to duration_sections.

If, after we add an edge that corresponds to the ending edge of a link duration, there are no open start edges (a start edge that belongs to a link, that does not yet have an accompanying end edge that belongs to that link):
Count the NEXT start edge as the start edge of the next duration section. This will prevent empty space caused by no link being open at that time from being counted towards the total.

Total up the time for all durations to get the total amount of time spent on links.
</pre>

The following diagram shows an example of what links could look like prior to our algorithm running:
[[File:Final_documentation_before_algorithm.PNG | center]]

After our algorithm runs, one can see that the overlapping segments are eliminated. For convenience, each duration_section is highlighted:
[[File:Final_documentation_after_algorithm.PNG | center]]

== ''' Test Plan ''' ==
=== Automated Testing Using RSpec ===
We have made changes to the controllers/submission_viewing_events_controller.rb and added test cases for the same, which have passed all the test cases. The tests can be executed <tt>rpec spec</tt> and the results for the files we have modified are shown below:

<pre>
Randomized with seed 20933
.....

Finished in 10.31 seconds (files took 7.85 seconds to load)
5 examples, 0 failures
</pre>

=== Coverage ===
In theory, the coverage of our overall project should increase significantly as we'll be adding tests that account for nearly a thousand lines of new code.

=== Manual UI Testing ===
We hope to not need any manual UI testing, though if it is needed, it'll look something like this:

Verifying Time is Tracked Correctly:
# Sign in as as a student
# Review an assignment
# Go through the review as normal
## Make sure you open links to github or pull requests
## Make sure you download and/or view files that are attatched
# Submit your review when you're done
# Log out as a student
# Log in as an instructor
# Navigate to the review report tab
# You should see a new column detailing the time the user spent on the review.

== ''' Our Work ''' ==
The code we created can be found below.
*[https://github.com/rohanpillai20/expertiza GitHub Repository]
*[https://github.com/expertiza/expertiza/pull/1607 Pull Request]
*[http:// Deployed URL] (Currently, Does not Exist)

The project could be run locally by cloning the [https://github.com/rohanpillai20/expertiza GitHub Repository] and then running the following commands sequentially.

<pre>
bundle install
rake db:create:all
rake db:migrate
rails s
</pre>

== ''' Team Information ''' ==
#Dylan Spruill (drspruil)
#Forrest Devita (fcdevita)
#Rohan Pillai (rspillai)
#Shalin Rathi (sjrathi)

Mentor: Akanksha Mohan (amohan7)
 Professor: Dr. Edward F. Gehringer (efg)

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://relishapp.com/rspec RSpec Documentation]

File:Final documentation after algorithm.PNG

2019-12-05T20:43:40Z

Fcdevita:

File:Final documentation before algorithm.PNG

2019-12-05T20:43:21Z

Fcdevita:

CSC/ECE 517 Fall 2019 - E1989. Track the time students look at other submissions

2019-12-05T20:42:51Z

Fcdevita:

== ''' Introduction ''' ==
The [http://expertiza.ncsu.edu/ Expertiza] project takes advantage of peer-review among students to allow them to learn from each other. Tracking the time that a student spends on each submitted resources is meaningful to instructors to study and improve the teaching experience. Unfortunately, most peer assessment systems do not manage the content of students’ submission within the systems. They usually allow the authors submit external links to the submission (e.g. GitHub code / deployed application), which makes it difficult for the system to track the time that the reviewers spend on the submissions.

== ''' Problem Statement ''' ==
Knowing how much time a student spends on a review is helpful when determining the quality of the review itself. That being said Expertiza needs to be able to '''track''' and '''display''' how much time a student spends on each review.

The time spent on a review is the sum of multiple sources:

# The time spent on the Expertiza assignment review page itself
# The time spent looking at external pages linked from the review page
# The time spent looking at downloadable files submitted by other students

The purpose of this project three-fold:

# Gather the timing data from the sources above.
# Display the data on the "Review Report" page (views/review_mapping/_review_report.html.erb) to show the time the student spent per review. Given that previous attempts have been rejected due to UI issues, it is important that this data is displayed cleanly.
# Write tests that confirm that the above goals are functioning correctly.

=== Current Implementation ===
So far, Expertiza does not have any such feature. However, three teams have already worked on this in the past but their builds were not merged due to some problems.
#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2016/E1705._Tracking_the_time_students_look_at_the_others%27_submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement Wiki])
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])

== ''' Proposed Solution ''' ==
After investigating the prior attempts at this task, we have decided it would be best to begin our project by building off of the work done in project [https://github.com/expertiza/expertiza/pull/1124 E1791]. The reason for choosing to build off of this particular project is because they have already put the work into tracking time spent viewing external pages as well as time spent viewing certain types of downloadable files. In order to achieve our goals outlined in the [[#Problem Statement|Problem Statement]] section, the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*This will be done by using onFocus and onBlur mouse events to tell when a user is actively working on the page.
*To prevent the user from cheating the system by just keeping the review page open without doing work, a timeout feature will be implemented. After 5 minutes of mouse/keyboard inactivity, a popup will be displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page will stop being tracked until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links and downloadable files will need to be made more accurate.
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Changes need to be made so that when the student is working on the Expertiza assignment review page time is not tracked for the external links or downloadable files.

; The overall time spent on the review needs to be displayed in a table on the "Review Report" page.
*The table will look like the following table, but with one modification. The entries in "Time per review in mins" will be clickable. When clicked, they will display a popup that contains detailed information on where the time for the review was spent. For example, if a student spent a total of 22 minutes on the review, it will show that the student spent 5 minutes on the Expertiza review page, 10 minutes looking at external links, and 7 minutes looking at downloadable files. It will display these details in text format as well as graphically using a bar graph. The purpose of choosing this design is two-fold:
# It will not require the instructor to have to go to a different page every time they want more details on a particular review. Given that the review report page takes a substantial time to load, this is a necessity.
# It will prevent the table on the review report page from being cluttered by figures and too much data.

[[File:Review report view.png]]

=== Design pattern ===
'''Strategy:''' In this project, for each review page, we need to deal with different type of links. For online links, such as github repo, youtube link, etc, we use javascript to open a new window detect time when the window is closed. For submitted files, such as txt, jpg, we will open it with views/response/_submitted_files.html.erb. For other files that need to be download and open locally, we can only make approximation for review time.

=== ''' Flowchart ''' ===

Reviewer 
Every time a start time is logged for expertiza/link/file, a new entry will be created in the database.End time will be updated on the last entry present for the same link/file.
[[File:E1989_Student_flowchart.png | center]]
 
Instructor 
Time for individual links will summed and then displayed as bar graph.
[[File:E1989_Instructor_flowchart.png | center]]

== ''' Code Changes ''' ==
1. Created a controller to handle the start/end times of the files viewed by the user. => '''controllers/submission_viewing_events_controller.rb''' 

2. Created a table in database to log start and end time for each link/file. It's schema is described below. 
{| class="wikitable"
|-
! Field
! Type
! Null
! Key
|-
| id
| int(11)
| NO
| Primary Key
|-
| map_id
| int(11)
| YES
|
|-
| round
| int(11)
| YES
|
|-
| link
| varchar(255)
| YES
|
|-
| start_at
| datetime
| YES
|
|-
| end_at
| datetime
| YES
|
|-
| created_at
| datetime
| NO
|
|-
| updated_at
| datetime
| NO
|
|}
3. Added view that will display with pie chart when instructor wants to see time spent on individual links/files. 

=== ''' Files that have been Added or Changed ''' ===
'''submission_viewing_events_controller.rb '''
The <tt>record_start_time</tt> function starts the time for links that have been visited by the user.
<pre>
def record_start_time
param_args = params[:submission_viewing_event]
# check if this link is already opened and timed
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: param_args[:map_id], round: param_args[:round], link: param_args[:link])
# if opened, end these records with current time
if submission_viewing_event_records
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
# time_record.update_attribute('end_at', start_at)
time_record.destroy
end
end
end
# create new response time record for current link
submission_viewing_event = SubmissionViewingEvent.new(submission_viewing_event_params)
submission_viewing_event.save

#if creating start time for expertiza update end times for all other links.
if param_args[:link]=='Expertiza Review'
params[:submission_viewing_event][:link] = nil
params[:submission_viewing_event][:end_at] = params[:submission_viewing_event][:start_at]
record_end_time()
end
render nothing: true
end
</pre>
 The <tt>record_end_time</tt> function records the end time for links that have been visited by the user.
<pre>
def record_end_time
data = params.require(:submission_viewing_event)
if data[:link].nil?
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], end_at: nil).where.not(link: "Expertiza Review")
else
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round], link: data[:link])
end
submission_viewing_event_records.each do |time_record|
if time_record.end_at.nil?
time_record.update_attribute('end_at', data[:end_at])
# break
end
end
respond_to do |format|
format.json { head :no_content }
end
end
</pre>
 The <tt>mark_end_time</tt> function records the end time for links that have no end times.
<pre>
def mark_end_time
data = params.require(:submission_viewing_event)
@link_array = []
submission_viewing_event_records = SubmissionViewingEvent.where(map_id: data[:map_id], round: data[:round])
submission_viewing_event_records.each do |submissionviewingevent_entry|
if submissionviewingevent_entry.end_at.nil?
@link_array.push(submissionviewingevent_entry.link)
submissionviewingevent_entry.update_attribute('end_at', data[:end_at])
end
end
respond_to do |format|
format.json { render json: @link_array }
end
end
</pre>

reviewer_details_popup.html.erb 
_review_report.html.erb 
_review_submissions_time_spent.html.erb 
response_times.coffee 
response_times.scss 
response_controller.rb 
response_times_controller.rb 
responses_times_helper.rb 
review_mapping_helper.rb 
response_time.rb 
_submitted_files.html.erb

=== ''' Algorithm For Precise Timing Accuracy ''' ===
Previous implementations attempted to record the time users spent viewing each link, but their solution has multiple problems. The primary problem was that when multiple links were open at the same time (thus overlapping with one another) the times at which they overlap are recorded twice. To fix this, we implemented the following algorithm within \app\views\reports\_review_submissions_time_spent.html.erb:

<pre>
Consider that each link duration (the amount of time that link is being looked at) has a starting and ending edge (see diagram below). These durations are stored in an array durations_to_consider.

Iterate through every link duration in durations_to_consider:
If any other link's start and end times both fit within our start and end time:
Remove from durations_to_consider

For list of durations_to_consider, add those durations' start and end edges to edge_array.

Ensure that entries in edge_array are sorted from earliest to latest.

Create list duration_sections.

Start at the first edge in edge_array. Go until next edge. Record the time difference as a duration and add time to duration_sections.

If, after we add an edge that corresponds to the ending edge of a link duration, there are no open start edges (a start edge that belongs to a link, that does not yet have an accompanying end edge that belongs to that link):
Count the NEXT start edge as the start edge of the next duration section. This will prevent empty space caused by no link being open at that time from being counted towards the total.

Total up the time for all durations to get the total amount of time spent on links.
</pre>

== ''' Test Plan ''' ==
=== Automated Testing Using RSpec ===
We have made changes to the controllers/submission_viewing_events_controller.rb and added test cases for the same, which have passed all the test cases. The tests can be executed <tt>rpec spec</tt> and the results for the files we have modified are shown below:

<pre>
Randomized with seed 20933
.....

Finished in 10.31 seconds (files took 7.85 seconds to load)
5 examples, 0 failures
</pre>

=== Coverage ===
In theory, the coverage of our overall project should increase significantly as we'll be adding tests that account for nearly a thousand lines of new code.

=== Manual UI Testing ===
We hope to not need any manual UI testing, though if it is needed, it'll look something like this:

Verifying Time is Tracked Correctly:
# Sign in as as a student
# Review an assignment
# Go through the review as normal
## Make sure you open links to github or pull requests
## Make sure you download and/or view files that are attatched
# Submit your review when you're done
# Log out as a student
# Log in as an instructor
# Navigate to the review report tab
# You should see a new column detailing the time the user spent on the review.

== ''' Our Work ''' ==
The code we created can be found below.
*[https://github.com/rohanpillai20/expertiza GitHub Repository]
*[https://github.com/expertiza/expertiza/pull/1607 Pull Request]
*[http:// Deployed URL] (Currently, Does not Exist)

The project could be run locally by cloning the [https://github.com/rohanpillai20/expertiza GitHub Repository] and then running the following commands sequentially.

<pre>
bundle install
rake db:create:all
rake db:migrate
rails s
</pre>

== ''' Team Information ''' ==
#Dylan Spruill (drspruil)
#Forrest Devita (fcdevita)
#Rohan Pillai (rspillai)
#Shalin Rathi (sjrathi)

Mentor: Akanksha Mohan (amohan7)
 Professor: Dr. Edward F. Gehringer (efg)

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://relishapp.com/rspec RSpec Documentation]

CSC/ECE 517 Fall 2019 - E1989. Track the time students look at other submissions

2019-11-10T21:33:56Z