Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:33:12Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good reference for completion of this project's requirements. This build has already implemented necessary functionality that helps track of time spent viewing external pages. The code also has an existing user-friendly UI. The reason it was not integrated into expertiza was due to large amounts of white space in files, confusing commit history, and complicated merge of E1791's code, who's repository fork is no longer existing. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], many changes needed to be manually implemented using a new commit history. We were able to add the following functionality, while reducing project size from 5000 line changes to less than 1000 line changes:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

* Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

* Added a pie chart to show breakdown of where time was spent during the review.
[[File:Pie_chart.png| center |Image: 50 pixels]]

* Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.
[[File:Popup.PNG|center|Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:32:44Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good reference for completion of this project's requirements. This build has already implemented necessary functionality that helps track of time spent viewing external pages. The code also has an existing user-friendly UI. The reason it was not integrated into expertiza was due to large amounts of white space in files, confusing commit history, and complicated merge of E1791's code, who's repository fork is no longer existing. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], many changes needed to be manually implemented using a new commit history. We were able to add the existing functionality, while reducing project size from 5000 code changes to less than 1000 lines.

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

* Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

* Added a pie chart to show breakdown of where time was spent during the review.
[[File:Pie_chart.png| center |Image: 50 pixels]]

* Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.
[[File:Popup.png|center|Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

File:Popup.PNG

2020-04-24T18:32:05Z

Hhleroy:

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:31:04Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good reference for completion of this project's requirements. This build has already implemented necessary functionality that helps track time spent viewing external pages. The reason it was not integrated into expertiza was due to large amounts of white space in files, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], many changes needed to be manually implemented using a new commit history. We were able to add the existing functionality, while reducing project size from 5000 code changes to less than 1000 lines.

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

* Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

* Added a pie chart to show breakdown of where time was spent during the review.
[[File:Pie_chart.png| center |Image: 50 pixels]]

* Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:30:40Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

* Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

* Added a pie chart to show breakdown of where time was spent during the review.
[[File:Pie_chart.png]]

* Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

File:Pie chart.png

2020-04-24T18:29:25Z

Hhleroy: Hhleroy uploaded a new version of File:Pie chart.png

Pie Chart that the instructor views.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:28:59Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

* Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

* Added a pie chart to show breakdown of where time was spent during the review.

* Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:28:45Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

# Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table

# Added a pie chart to show breakdown of where time was spent during the review.

# Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:28:33Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

# Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table
# Added a pie chart to show breakdown of where time was spent during the review.
[[File:Example.jpg]]
# Added a timeout reminder that asks the student if they are still performing the review after 5 minutes of the computer idling.

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:27:49Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

# Added time value to the table on "Review Reports" page. Found in "Team Reviewed" column of the table
# Added a pie chart to show breakdown of where time was spent during the review.
[[File:Example.jpg]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:25:18Z

Hhleroy: /* UI Changes */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be implemented:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits, confusing commit history, and complicated merge of E1791's code. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be manually implemented:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section shows various alterations to the Expertiza user interface in order to display information about time spent on a given review. Additionally, some alterations were made to the student's user interface to assist them in their review.

#

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:20:05Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits and unsatisfactory commit messages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''UI Changes''' ==
The following section displays

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T18:18:59Z

Hhleroy: /* Proposed Solution */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Implemented Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits and unsatisfactory commit messages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created (6)''' ===
<hr/>

'''controllers/submission_viewing_events_controller.rb'''

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

'''spec/controllers/submission_viewing_events_controller_spec.rb'''

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

'''db/migrate/20200420201807_create_submission_viewing_events.rb'''

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

'''app/views/submission_viewing_events/index.html.haml'''

Created a simple index page for viewing of submission events.

'''app/views/submission_viewing_events/edit.html.haml'''

Created file to allow for editing capabilities of submission viewing events.

'''app/views/submission_viewing_events/_form.html.haml'''

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited (18)''' ===
<hr/>

'''db/schema.rb'''

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

'''config/routes.rb'''

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

'''app/views/submission_viewing_events/new.html.haml'''

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/views/submission_viewing_events/show.html.haml'''

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

'''app/helpers/submitted_content_helper.rb'''

Updated line 32 to incorporate file link for the return variable, ret.

'''app/views/popup/view_review_scores_popup.html.erb'''

Break up query in line 55 into two queries.

'''app/views/reports/_review_submissions_time_spent.html.erb'''

Created a view for reports controller.

'''app/views/reports/_review_report.html.erb'''

Made changes to _review_report to reflect changes in reports_controller

'''app/views/reports/_review_submissions_time_spent.html.erb'''

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

'''app/views/reports/_review_report.html.erb'''

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

'''app/views/response/_submitted_files.html.erb'''

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

'''app/views/response/view.html.erb'''

Edited to set submission check variable to false.

'''app/views/response/response.html.erb'''

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

'''app/views/submitted_content/_main.html.erb'''

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

'''app/views/submitted_content/_hyperlink.html.erb'''

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

'''app/views/submitted_content/_submitted_files.html.erb '''

Added embedded function to .html file for tracking viewing of windows in the browser.

'''app/controllers/response_controller.rb'''

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

'''app/controllers/tree_display_controller.rb '''

Added a catch for errors on a function call in tree_display_controller, a file not directly related to the project. This was necessary for handling Travis CI error upon initial commit.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T17:56:40Z

Hhleroy: /* Manual Testing */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits and unsatisfactory commit messages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''db/migrate/20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

==== '''app/views/submission_viewing_events/index.html.haml''' ====

Created a simple index page for viewing of submission events.

==== '''app/views/submission_viewing_events/edit.html.haml''' ====

Created file to allow for editing capabilities of submission viewing events.

==== '''app/views/submission_viewing_events/_form.html.haml''' ====

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

==== '''spec/controllers/submission_viewing_events_controller_spec.rb''' ====

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

==== '''app/views/submitted_content/_submitted_files.html.erb ''' ====

Added embedded function to .html file for tracking viewing of windows in the browser.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
* Log in as instructor
::Username: Insturctor6
::Password: password
* Click on "Manage..." and select "Assignments" in the drop down list
* Navigate to "Test Timetracking (FINAL)" assignment
* Click on "View Report" Icon
* Ensure drop down reads "Review Report" and click "View" button
* This will display a table with information about reviews for the assignment
* Select one of the reviews and click on the time value hyperlink
* This should display a pie chart and table with a break down of time spent on review
* Click on "Manage..." and select "Impersonate User" in the drop down list
* Enter "student74XX" where the XX can range from 90-99
* Navigate to "Assignments" tab
* Select "Test Timetracking (Final) review"
* Complete the review and write down about how the it took
* Click the "Revert" button in the header to go back to an instructor
* Navigate back to the "Review Report" table
* Confirm appropriate entry was added to the table and matches, or is near, the personally recorded time

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T17:45:29Z

Hhleroy: /* Manual Testing */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits and unsatisfactory commit messages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''db/migrate/20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

==== '''app/views/submission_viewing_events/index.html.haml''' ====

Created a simple index page for viewing of submission events.

==== '''app/views/submission_viewing_events/edit.html.haml''' ====

Created file to allow for editing capabilities of submission viewing events.

==== '''app/views/submission_viewing_events/_form.html.haml''' ====

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

==== '''spec/controllers/submission_viewing_events_controller_spec.rb''' ====

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
:Username: Insturctor6
:Password: password
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T17:44:14Z

Hhleroy: /* Our Implementation */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1662/commits E1989] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages and the reason it was not integrated into expertiza was due to the amount of white space in commits and unsatisfactory commit messages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] is used, however small edits were made to provide clean code and a trackable commit history. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved, the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use '''map_id''' to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''db/migrate/20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

==== '''app/views/submission_viewing_events/index.html.haml''' ====

Created a simple index page for viewing of submission events.

==== '''app/views/submission_viewing_events/edit.html.haml''' ====

Created file to allow for editing capabilities of submission viewing events.

==== '''app/views/submission_viewing_events/_form.html.haml''' ====

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

==== '''spec/controllers/submission_viewing_events_controller_spec.rb''' ====

Implemented test cases for created SubmissionViewingEventsController using Rspec as per expertiza standard.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo] 
[https://github.com/rohanpillai20/expertiza/tree/E1989_Merge_Fixes E1989 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:] 
[https://github.com/expertiza/expertiza/pull/1728/commits Pull commits] 
[https://www.youtube.com/watch?v=vyidgf4sjcg Video Demonstartion]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T00:44:57Z

Hhleroy: /* app/views/submission_viewing_events/index.html.haml */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to better suite our proposed implementation. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use map_id to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

==== '''app/views/submission_viewing_events/index.html.haml''' ====

Created a simple index page for viewing of submission events.

==== '''app/views/submission_viewing_events/edit.html.haml''' ====

Created file to allow for editing capabilities of submission viewing events.

==== '''app/views/submission_viewing_events/_form.html.haml''' ====

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T00:44:10Z

Hhleroy: /* Files Created */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to better suite our proposed implementation. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use map_id to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

==== '''app/views/submission_viewing_events/index.html.haml''' ====

Created a simple index page for viewing of submission events.

==== '''app/views/submission_viewing_events/edit.html.haml''' ====

Created file to allow for editing capabilities of submission viewing events.

==== '''app/views/submission_viewing_events/_form.html.haml''' ====

Created partial form that was used to fill in edit.html.haml file, as well as a few others.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T00:36:49Z

Hhleroy: /* Files Edited */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to better suite our proposed implementation. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use map_id to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

=== '''Files Edited''' ===
<hr/>

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/response/view.html.erb''' ====

Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

Edited to contain two functions to track start and end time or review. Added hidden field tags to form to track timeouts, as well as a submission check to ensure reviewer is still working on review.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T00:33:45Z

Hhleroy: /* app/views/response/view.html.erb */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to better suite our proposed implementation. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use map_id to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

=== '''Files Edited''' ===
<hr/>
==== '''app/views/response/view.html.erb''' ====
Edited to set submission check variable to false.

==== '''app/views/response/response.html.erb''' ====

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-24T00:22:49Z

Hhleroy: /* Files Edited */

__TOC__
== '''Abstract''' ==
Expertiza is an open source software project created using Ruby on Rails. Expertiza allows instructors to craft new assignments and edit existing ones. This flexibility ensures that each round of students gets an experience that is appropriate to the given situation. It also allows the instructor to create a list of project options and have the students bid for their favorite project.

While their are a plethora of benefits for instructors, students also gain some benefits when using Expertiza. They are able to form teams and keep track of the past peers they have worked with, and are also able to manage the progress and submission of their assignments.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review is most important. One can track the amount of time spent on the review by tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a resource is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and resources should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the completed aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://github.com/expertiza/expertiza/pull/1662/commits E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.
*There are a few solutions one could implement to fix such an issue, such as marking both the start time and end time for when an external link or application was accessed by a user. Another solution which we will attempt in our first iteration of development is to track the time that an external link was clicked, and use the submission time of the review as our estimated end time for the external link. We thought of this design choice because once the report is submitted, the access time to external links should be stopped since the review is complete.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints about the use of a tabular method that was implemented in [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791], stating that the review report table becomes too cluttered, we intend to create a pop-up window that will display the results in a table or graph of some sort. The proposed pop up will display all necessary information in a neat and simple self contained form. The exact display is undetermined, however proposed solutions include another tabular design, bar graphs, pie charts, etc.

== '''User Stories''' ==
The following section outlines the predicted user interactions with our software. Two potential users have been identified; users and instructors.

====Student====
Students will be interacting with the implementation when filling out a review. A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to better suite our proposed implementation. Initially, the user will click on the review they want to complete. Once the link is clicked the time will start to be logged. Upon the clicking of an external link another another timer will begin tracking it. This process repeats until the submission is saved/submitted or the page is exited out of. In the case where a review was previously saved the timer will pick up from the last tracked time. The diagram below displays the flow of interactions visually.
[[File:User_Story.png| center |Image: 50 pixels]]

====Instructor====
Instructors will be interacting with the implementation when the are observing student reviews. Once again, a similar story from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for the instructor with minor changes. Firstly, the instructor will navigate to a page that displays all current and previously reviews. They will then be able to select a review from the given list and look at statistics about resources accessed and time spent on each resource. Saved reviews and already submitted reviews will be tagged differently. The diagram below displays the flow of interactions visually.

[[File:Instructor_Story.png | center |Image: 50 pixels]]

== '''Database Schema''' ==
Our datatable is named '''submission_viewing_events'''. The attributes and their descriptions are provided in the spreadsheet below. We use map_id to join '''response_maps''' table for storing the primary information. '''Round''' contains an int variable associated with the specific round review, '''link''' contains a character string storing the external link the reviewer clicked on, '''start_at''' contains the starting time the external link was pressed, while '''end_at''' contains the time when the link was closed.

[[File:Db_FINAL.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The highlight demonstrate which field correspond to response_maps key.

[[File:Db_schema_FINAL.png | center]]

== '''Code Changes''' ==
Our implementation builds off of the work done in E1989. This previous project has a substantial amount of the functionality already completed for tracking and viewing external pages and certain types of files. The primary error in this implementation is that when multiple files are open, the times at which they record reviews are overlapped creating inaccurate review reports. The displayed review report will also need to be edited to present review reports in a visually appealing manner. The following changes were made:

=== '''Files Created''' ===
<hr/>

==== '''controllers/submission_viewing_events_controller.rb''' ====

Creates a controller for handling events of submission viewing reviews. Created using rails 'generate controller' function. Includes three methods

# record_start_time : function starts the time for links that have been visited by the user.
# record_end_time : records the end time for links that have been visited by the user.
# mark_end_time : records the end time for links that have no end times.

==== '''20200420201807_create_submission_viewing_events.rb''' ====

Migration file to create data table for handling log start and end time for each link/file in the review. Created using rails 'generate model' function. More info regarding schema is included in the Database section.

=== '''Files Edited''' ===
<hr/>
==== '''app/views/response/view.html.erb''' ====

==== '''app/views/response/response.html.erb''' ====

==== '''db/schema.rb''' ====

Updated to include the SubmissionViewingEvents model migration, as executed via a 'rake db:migrate"

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

will be edited to accurately log viewing times for multiple links open at the same time. This will include creating methods such as calculateTime to help with parsing review times of different links, and a function drawChart to display review times in a visually appealing manner.

==== '''app/views/reports/_review_report.html.erb'''====

will be edited to reflect chart made in _review_submissions_time_spent.html.erb

==== '''app/views/submission_viewing_events/new.html.haml'''====

Created new.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/views/submission_viewing_events/show.html.haml'''====

Created show.html.haml for submission_viewing_events for assisting in setting up the views.

==== '''app/helpers/submitted_content_helper.rb'''====

Updated line 32 to incorporate file link for the return variable, ret.

===='''app/views/popup/view_review_scores_popup.html.erb'''====

Break up query in line 55 into two queries.

==== '''app/views/reports/_review_submissions_time_spent.html.erb'''====

Created a view for reports controller.

==== '''app/views/reports/_review_report.html.erb'''====

Made changes to _review_report to reflect changes in reports_controller

==== '''app/views/response/_submitted_files.html.erb''' ====

Added JavaScript to this file which allows tracking of start and end times of accessing a hyperlink. Most functionality is on-click for tracking these times.

==== '''config/routes.rb''' ====

Added collection for start and end times when hyperlink is accessed. Includes post and get calls for recording start and end times as necessary (see lines 420-428)

==== '''app/controllers/response_controller.rb''' ====

Added field @current_round to file for tracking the current round of reviews for a student. See lines 83 and 267 for variable instantiation.

==== '''app/views/submitted_content/_main.html.erb''' ====

Added checks for local assignment submissions, where if true a partial specifically for hyperlinks and submitted files are loaded. See lines 10-20 for specific changes.

==== '''app/views/submitted_content/_hyperlink.html.erb''' ====

Adds functionality to ensure that user must click on a hyperlink instead of copying and pasting the link. From line 41-end there is more JavaScript that marks start and end times for accessing these hyperlinks.

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. The primary controller for testing will be controllers/submission_viewing_events_controller.rb, as edits will be made to mark_end_time and record_end_time methods to ensure that recorded review times for resources are not overlapping.

Proposed Rspec Tests:
* When a new review is started probe the Expertiza timer and ensure it returns the start value (0)
* When a saved review is started again probe the Expertiza timer and ensure it returns the time value stored in the database
* When an external link is clicked on a new review probe the timer link for that link and ensure it returns the start value (0)
* When an external link is clicked on a previously saved review probe the timer link for that link and ensure it returns and ensure it returns the time value stored in the database
* When a review is saved/submitted ensure that the time sent to the database matches time stored in database after execution
* When a review is clicked on by the instructor the data returned from date base should match test data.

===Manual Testing===
# Log in as instructor
# Assign a review to a student
# Log in as student
# Complete the dummy review in 5 minutes
# Log in as instructor
# Navigate to review report tab
# Verify that the review was completed ~5 minutes.

== '''References''' ==

=== Useful Links ===

[https://github.com/expertiza/expertiza Expertiza Repo] 
[https://github.com/rohanpillai20/expertiza E1791 Repo]

=== Our Implementation ===

[https://github.com/jamcdon3/expertiza/tree/E2015 Github Repo:] 
[https://github.com/rohanpillai20/expertiza/pull/51 Pull Request:]

=== Team Information ===
'''Mentor:''' 
Abhirav Kariya

'''Students:''' 
Hartley Leroy 
John McDonald 
Christian Morris 
Tyler Sattler

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T01:03:05Z

Hhleroy: /* Rspec */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T01:01:35Z

Hhleroy: /* Rspec */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T00:23:41Z

Hhleroy: /* Design pattern */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T00:11:55Z

Hhleroy: /* Proposed Solution */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T00:06:11Z

Hhleroy: /* Proposed Solution */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-14T00:00:43Z

Hhleroy: /* Student */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:55:36Z

Hhleroy: /* User Stories */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:41:16Z

Hhleroy: /* User Stories */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:40:35Z

Hhleroy: /* User Stories */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:32:49Z

Hhleroy: /* User Stories Flow Charts */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:32:27Z

Hhleroy: /* User Stories Flow Charts */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:30:54Z

Hhleroy: /* User Stories Flow Charts */

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:24:13Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Enter abstract here.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.

Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
=== User Stories Flow Charts ===
 
[[File:User_Story.png |Image: 50 pixels]] [[File:Instructor_Story.png |Image: 50 pixels]]
 

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:23:10Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Enter abstract here.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

== User Stories ==
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
 
[[File:User_Story.png |Image: 50 pixels]] [[File:Instructor_Story.png |Image: 50 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:22:16Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Enter abstract here.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png |Image: 50 pixels]][[File:Instructor_Story.png |Image: 50 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:20:45Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Enter abstract here.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | center |Image: 100 pixels][File:Instructor_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:20:32Z

Hhleroy: /* User Stories */

__TOC__
== '''Abstract''' ==
Enter abstract here.

== '''Problem Statement''' ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

== '''Previous Implementations''' ==
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== '''Proposed Solution''' ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | center |Image: 100 pixels]][[File:Instructor_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== '''Testing Plan''' ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:14:49Z

Hhleroy: /* User Stories */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png,Instructor_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:13:51Z

Hhleroy: /* User Stories */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | left |Image: 100 pixels]][[File:Instructor_Story.png | right |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T23:13:12Z

Hhleroy: /* User Stories */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

=== User Stories ===
Student 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Edited ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_.PNG | center]]

 

Below shows the database relationship with other tables. '''submission_viewing_events''' will be touching '''response_maps'''. The boxes demonstrate which fields correspond to each other.

[[File:Db_schema1.png | center]]

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T18:08:51Z

Hhleroy: /* Design pattern */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

User 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:Instructor_Story.png | center |Image: 100 pixels]]

=== Files to be Added and Editted ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_schema.PNG | center]]

 

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

File:Instructor Story.png

2020-04-13T18:08:20Z

Hhleroy:

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T17:52:05Z

Hhleroy: /* Design pattern */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

User 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.
[[File:User_Story.png | center |Image: 100 pixels]]
Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:E1989_Instructor_flowchart.png | center |Image: 100 pixels]]

=== Files to be Added and Editted ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_schema.PNG | center]]

 

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

File:User Story.png

2020-04-13T17:50:58Z

Hhleroy:

File:User Story .png

2020-04-13T17:50:10Z

Hhleroy:

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T17:49:48Z

Hhleroy: /* Design pattern */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

User 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.

Instructor 
Once again, a similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:E1989_Instructor_flowchart.png | center |Image: 100 pixels]]

=== Files to be Added and Editted ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_schema.PNG | center]]

 

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T17:49:32Z

Hhleroy: /* Design pattern */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

User 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implementation.

Instructor 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:E1989_Instructor_flowchart.png | center |Image: 100 pixels]]

=== Files to be Added and Editted ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_schema.PNG | center]]

 

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-13T17:49:13Z

Hhleroy: /* Design pattern */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. It is important that functionality be added so Expertiza can track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

The overall amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened untill when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

The following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature. Previous attempts to implement this functionality have been made but the work was not merged into Expertiza code base for various reason. These previous projects are summarized below:

#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both. Tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.
User 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] is used, however small edits were made to the diagram to better suite our proposed implmnetation.

Instructor 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:E1989_Instructor_flowchart.png | center |Image: 100 pixels]]

=== Files to be Added and Editted ===

# Create a controller to handle the start/end times of the files viewed by the user in => controllers/submission_viewing_events_controller.rb
# Create a datatable to handle log start and end time for each link/file. Schema for this table is provided in the below section.
# Edit and add views to display results in a user-friendly manner. Edits to views should be defined in both the wiki and through modular commits on the repository in order to clearly distinguish code changes, an issue that prevented [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] from pushing to base.

=== Database Schema ===

Our datatable will be named '''submission_viewing_events'''. The attributes and their descriptions are provided in the below spreadsheet. There is some uncertainty in how the time will be tracked separately for viewing downloadable files vs viewing the Expertiza assignment review page. Final form of the data table with this tracking feature will be elaborate on in future submissions.

[[File:Db_schema.PNG | center]]

 

== Testing Plan ==
===Rspec===
Automatic testing will be achieved using Rspec. We will test the changes made to controllers/submission_viewing_events_controller.rb to ensure proper test converge. Test cases are to be written in a meaningful and elegant manner, adhering to the following rules:

* Test incoming query messages by making assertions for what they send back
* Test incoming command messages by making assertions about direct public side effects
* Do not test private methods
* Do not test outgoing query messages
* Expect to send outgoing command messages

===Manual Testing===
1. Log in as instructor

2. Assign a review to a student

3. Log in as student

4. Complete the dummy review in 5 minutes

5. Log in as instructor

6. Navigate to review report tab

7. Verify that the review was completed ~5 minutes.

CSC/ECE 517 Spring 2020 / E2023 Track the Time Students Look at Other Submissions

2020-04-08T02:21:26Z

Hhleroy: /* Proposed Solution */

__TOC__
== Problem Statement ==
Understanding how much time a student spends reviewing another's work is beneficial in order to better estimate the quality of said review. Given that fact it is important that Expertiza is able to track and display the given amount of time a student spends on a review.

The time spent on each review is a summation of multiple sources:
* Time spent of the Expertiza review itself
* Time spent looking at external links
* Time spent looking at downloadable files

However, overall the amount of time directly spent on the review itself is most important. One can track the amount of time spent on the review itself quite easily by simply tracking the amount of time from when the page is opened till when the review is saved/submitted. Therefore, being able to track the time from once a link is opened till when the review is saved/submitted will provide a reasonable estimate of the amount of time spent on each resource. This has the benefit of only needing to track information interacted with on the Expertiza review page, as opposed to other external files and links.

Therefore, the following tasks need to be accomplished:
# Time spent on an Expertiza review must be tracked
# Time spent on external links and documents should be tracked/estimated
# Overall time spent on the review should be displayed in a "user friendly manner"

=== Current Implementation ===
Thus far, Expertiza does not have the aforementioned feature, however previous work was undertaken by other teams. The prior work of other teams was not merged into the Expertiza code base do to the following issues.
#[https://github.com/expertiza/expertiza/pull/875 E1705] identified how to track the active time of windows opened from the submitted links.
#[https://github.com/expertiza/expertiza/pull/1124 E1791] provided detailed insights on how they planned to track time taken by a student in viewing a submission and possible edge cases. Further, they also implemented popups and figured out a way to open downloadable files. However, the details are rendered in a not-so-friendly manner and hence it was not merged.
#[https://github.com/expertiza/expertiza/pull/1309 E1872] tried to solve this by incorporating the statistics in the review reports page, but their UI made the page cluttered and not friendly. Further, it was hard to identify which statistic belonged to which review, and there were almost no tests. ([http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1872_Track_Time_Students_Look_At_Other_Submissions Wiki])
#[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1989._Track_the_time_students_look_at_other_submissions E1989] tried to solve this by building off of [https://github.com/expertiza/expertiza/pull/1124 E1791]. The team took the base code and attempted to implement the ability to track time spent on the review page, as well as other external links, however the code was not merged due to a large amount of white-space, as well as difficulty in distinguishing actual code changes.

== Proposed Solution ==
In review of previous iterations of this project, it was found that project [https://github.com/expertiza/expertiza/pull/1124 E1791] would prove to be a good starting point for completion of this project's requirements. This build has already implemented systems that help track time spent viewing external pages. To achieve our goals outlined in the [[#Problem Statement|Problem Statement]], the following changes need to be made:

; The time spent on the Expertiza assignment review page needs to be tracked.
*Due to Expertiza generating report text boxes with HTML iFrames, we will track whether or not the document hasFocus() to determine when a student is on the page or not.
*After 5 minutes of mouse/keyboard inactivity, a popup is displayed asking if the user is still working. At that point, the time contributed towards the total by the Expertiza page is paused until the user interacts with the popup to indicate they are still working. This is already implemented in project [https://github.com/expertiza/expertiza/pull/1124 E1791].

; The time spent viewing the external links/downloadable files needs to be tracked or estimated
*Currently, if a student has an external link open as well as the Expertiza page, time is being tracked for both, however tracking of external links may be unnecessary for the project, and instead an estimation approach may be taken. Either way it is likely that this measurement will be altered throughout development of this project.

; The overall time spent on the review needs to be displayed in a "user friendly manner" on the "Review Report" page.
*Due to complaints on a tabular method, citing the review report table becoming too cluttered, we intend to create a pop-up window that will display the results in a table. Unlike the approach taken in project [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 from the fall of 2019], this project will work to display all information using the aforementioned tabular design in a pop-up window. With this approach, the table provides the information and estimates regarding all of the external links accessed throughout this review in a simple and uncluttered fashion.

=== Design pattern ===
'''Strategy:''' This project, for each review page, deals with different type of links. Javascript is used to track the access time for online links, such as github repo, youtube links, and more. For submitted files, such as txt, jpg, we will time it with views/response/_submitted_files.html.erb.

Instructor 
A similar workflow from [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1791._Track_the_time_that_students_look_at_the_other_submissions_-_logging_improvement E1791 Fall '19] will be used for an instructor to access the tabularized information regarding access times:
[[File:E1989_Instructor_flowchart.png | center]]