Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2021 - E2111. Refactor github metrics integration

2021-04-30T15:04:19Z

Jmhurst:

==Abstract==
Github repository and pull request links are submitted as part of programming projects in several NC State CSC courses. Instructors would like to compile statistics on these Github links on a per-project, per-team basis, and integrate display of these data within Expertiza (similar to Github Insights). Code was introduced to add this functionality in a [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2018-_Project_E1858._Github_metrics_integration 2018 project, E1858,] which was not merged, and another team re-visited [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1983._Refactor_E1858,_Github_metrics_integration_view this project in 2019], which was also not merged. The project team will troubleshoot existing code to eliminate issues that prevented these projects from being merged. Additionally, the team will add new features based on design patterns and MVC principles, including a Metrics model, that will decouple the Github Metrics functionality from other areas of the application and allow for future extensibility. Finally, the team will investigate DRYing the code using existing graphing functionality, and complete an analysis of the automated testing for existing and new code.

==Functionality==
The Github metrics system, as currently implemented, queries data from Github based on repository and pull request links submitted for the assignment. The commit frequency is displayed in a bar graph. Numeric statistics, as well as Travis CI status, are displayed in a table.

[[File:GithubMetricsExample.png]]

==Methodology==
The team has synced the revised 2019 project code with the current status of the Expertiza/beta branch and begun analyzing the existing code. Prior customer feedback has advised the team to address inextensibility/coupling issues, documentation issues, and synchronization issues. We are performing our own end-to-end analysis of the code in addition to this feedback, and have so far found numerous action items. We have separated these action items into Primary and Secondary objectives: Primary objectives are "must-haves" that need to be addressed before the existing code is mergeable. Secondary objectives are "nice-to-haves," or features that were either left out of the original project or would make the integration display more complete.

==Design Patterns==
In order to achieve the Primary Objectives while making the application more extensible, the team proposes using a more strict implementation of the '''Adapter''' design pattern to decouple the details of the Github API from the proposed Metrics model. Elements of the '''Observer''' design pattern will also come into play to ensure that once Github data is queried and stored, and an assignment has come to a close, the Metric for that Assignment is unsubscribed from the Github API. This will further decouple the dependency on API data while retaining the data for future statistical analysis and/or research studies.

==Team Action Items==
These action items include both refactoring tasks, and required feature enhancements to be implemented.

====Minor API Changes Breaking Functionality====
The Github API has had minor changes since 2018 which are causing the previously working code to either not display, or throw a NoMethodError. We have already analyzed and applied bugfixes to address these minor API changes, but would like to further refine how the API calls are set-up to decouple our code from the API and make it more robust.

One key issue is that the Github API no longer permits queries with empty hash keys. This shows up in queries with a single page, where <code> after: ___</code> has, in the past, been left blank. In order to resume queries, the team now tests whether <code> after: SHA1 </code> should be included in the query, and if not, we do not add the empty key. This permits data to be pulled from Github where repos have fewer than two pages of commits.

====Refactoring Improvements====
Several method and variable names do not use good Ruby style, and should be refactored to improve style and clarity. Some examples include get_pull_request_details() and authorize_github(). These shall be refactored for improved clarity, and may be relocated within the application to make the code better fit the MVC system. See Metrics Model section for further details.

====Improvements to Data Time and User Constraints====
If a team submits a repo containing merged data, with many Expertiza Development Team commits, these commits currently show up as part of the team metrics. Improved logic needs to be implemented to timebox the statistics to only cover commits made during the duration of the project. Additionally, improvements are needed for the way in which commit owners are handled, so that metrics are not stored/displayed/statistically analyzed for Expertiza Development Team commits. An example of a problematic submission is shown below.

[[File:GithubIssue1.png]]

====Major Feature: Metrics Model====
The Expertiza system architect has instructed the team to handle a major shortcoming in the original project as part of this body of work: the creation of a new model to handle local storage of the metric data. This new model should be able to support future extensibility for new metrics, like Travis CI or Danger. This change will bring the Github metrics system in full compliance with MVC principles and allow for local warehousing of data for performance and data retention.

The current implementation works synchronously, and each time a Github integration route is followed, the entire collection of Github data is requested from the Github API and used to render the graphs and statistics. These data are cumbersome, difficult to work with, and nearly impossible to extend to future functionality. Furthermore, this coupling to the Github API means that data is not being warehoused within the system and makes adding new functionality much more difficult. The code uses controller methods to load the data and format it for display.

The team proposes creating a fully realized MVC implementation to separate Github queries, warehousing of data, and displaying of graphs and statistics which are currently handled in a single chain of controller methods. This will require implementation of a new model, and additional database tables to store the data as it is queried. Also, views will need to be uncoupled from the Github API and reconfigured to render data from the local data warehouse. Furthermore, this will allow methods that aren't CRUDlike to be moved from the Github metrics controller into the new model.

'''Diagram of Proposed MVC Architecture for New Metrics Model''' 
[[File:githubModelFlow.png]]

One point of discussion among the team and our mentor will be, should an update query be run each time the page is displayed, or should a burst of queries be run at the expiration of a deadline to populate the database table? Another option may be to create a manual "Github Update" link, but reject all data after a deadline. Yet another option would be to keep all Github data and use the "late data" to call out number of late submissions on the statistics page.

=====Proposed Schema and Entries Example=====
User '''JoeCodeMonkey''', on '''team 15''', added 3 commits which were queried from Github. The ''Metrics'' table will keep track of each metric, with the ''Metrics_Data_Points'' table keeping track of each individual data point. The ''Metric_Sources'' and ''Metric_Data_Point_Types'' tables are used as lookup tables to keep track of the types of metric data being stored in ''Metrics'' and ''Metrics_Data_Point''.
 Table: Metric_Sources<table style="border-collapse:collapse;margin-left:13.275pt" cellspacing="0"><tr style="height:30pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> ID(PK)</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Source Name</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Description</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Github</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">CVS Hosting</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Travis CI</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Build Tool</td></tr></table> Table: Metric_Data_Point_Types<table style="border-collapse:collapse;margin-left:13.275pt" cellspacing="0"><tr style="height:56pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> ID(PK)</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Source_Id (FK)to Metric Sources Table)</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Value_Type</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Name</td><td style="width:144pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Description</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Github -0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">string</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">github_user</td><td style="width:144pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Github user</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">int</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">lines_added</td><td style="width:144pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Number of lines added</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">int</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">lines_removed</td><td style="width:144pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Number of lines removed</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">3</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">datetime</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">date_committed</td><td style="width:144pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Date of change</td></tr></table> Table: Metrics<table style="border-collapse:collapse;margin-left:13.275pt" cellspacing="0"><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">ID(PK)</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Source_Id (FK</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Team_ID (FK)</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0 -Github</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">15 - Team 15</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0 - Github</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">15 - Team 15</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0 - Github</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">15 - Team 15</td></tr></table> Table: Metrics_Data_Points<table style="border-collapse:collapse;margin-left:13.275pt" cellspacing="0"><tr style="height:45pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Id</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Metric_Id (FK to metric table)</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">Metric_Data_P oint_Type_Id (FK)</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt"> Value</td></tr><tr style="height:17pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">(metric id )0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0 (github_user)</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">JoeCodeMonkey</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1 (lines_added)</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">5</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2 (lines_remove</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">10</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">3</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">3 (date_commit</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1/2/2017 1600</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">4</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1 (metric id 1)</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">JoeCodeMonkey</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">5</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">8</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">6</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">5</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">7</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">3</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1/5/2017 1700</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">8</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">0</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">JoeCodeMonkey</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">9</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">7</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">10</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">6</td></tr><tr style="height:15pt"><td style="width:52pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">11</td><td style="width:68pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">2</td><td style="width:73pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">3</td><td style="width:116pt;border-top-style:solid;border-top-width:1pt;border-left-style:solid;border-left-width:1pt;border-bottom-style:solid;border-bottom-width:1pt;border-right-style:solid;border-right-width:1pt">1/20/2017 1430</td></tr></table>

====Lack of Comments / Opaque Code====
The majority of methods added in E1858 are lacking comments describing their purpose and functionality and comments were not modified for any modified methods. 
Comments will be added to the following files
<ul>
<li>app/controllers/auth_controller.rb: New methods oauth_login and github_login need comments, existing method action_allowed? needs a comment
<li>app/controllers/grade_controller.rb: Large amount of new methods need comments, existing method "view" needs modification to existing comment
<li>app/helpers/grades_helper.rb: The methods display_github_metrics, chart_options, and graph_scales need comments describing functionality
<li>app/views/assignments/list_submissions.html.erb: New GitHub Metrics column added to table
<li>app/views/grades/_tabbing.html.erb: Class "tmr_tab" will have to be explained
<li>app/views/grades/_teammate_reviews_tab.html.erb: Class "github" will have to be explained
<li>app/views/grades/view.html.erb: JavaScript functionality to be moved to a helper .js file and then commented.
<li>app/views/grades/view_github_metrics.html.erb : JavaScript functionality to be moved to a helper .js file and then commented.
<li>spec/controllers/grades_controller_spec.rb: Comments will have to be added describing the RSpec tests including their purpose and how they achieve their tests
</ul>

====DRYing Graphing Code====
The existing code uses new code to graph the statistical Github data. Code already exists elsewhere in Expertiza (Statistics) to graphically display statistical data, and the team intends to replace the Github Graphing code by re-using the existing graphing code, if possible.

====Properly Timebox Statistics====
The existing code pulls all commits from a repository or pull request. We propose utilizing the assignment start and end dates to ensure that only commits which apply to the scope of the project are included in the statistics view/analysis.

====Bugfix: Hide Expertiza Development Team====
At this time, if a team merges in many old commits by the Expertiza Development team, the bar graph statistics may be thrown off by including the development team ("Winbobob" et al). Logic exists in the code to prevent these developer's work from being credited to the project team, but it is not working reliably. If the data permanence objective is met, creating more complex queries should be easier, and the team would like to explore updating this feature to to "hide known contributors," and ensure that it is working reliably. This would hide Github data from any user included on the "blacklist" which can be manually curated with the short list of Expertiza developers. While this would not address edge cases where an Expertiza developer later participates in a course using Expertiza, this edge case should be quite rare.

==Test Plan==
The 2018 project wrote numerous tests, several of which are deeply coupled into the grades_controller tests. The team proposes several changes to the testing strategy.

====Decouple Tests / Pass Builds====
Builds never passed for this project because, in the current code, a user must be logged in and authorized to Github to render grades#view, and this broke several (expected) test cases. As part of decoupling Github from the grades controller, we expect to decouple and revert tests for the grades controller such that the tests become independent, permitting the build to pass if our Github controller tests pass as expected.

====Detailed Analysis of Github Testing====
The previous team wrote numerous intricate tests for the new functionality. Our team intends to go through these tests line-by-line and ensure that having so many tests is necessary, and the most elegant way to address the issue. We believe that some of the Github refactoring should move methods to become private, which may mean that specific tests for that method shouldn't be used. Alternately, particularly for API authentication which can be tricky, it may be prudent to have discrete tests. Finally, we want to ensure that the tests are actually covering the code -- we are receiving a TravisCI warning about tests without expectations, which needs to be resolved before the code can be merged.

====New Tests for Metrics Model====
The new model being created, along with associated database tables, will lead to new tests being required for this model. These tests will need to be written first in keeping with TDD principles. Also, some tests may need revision when code is moved from the Github Metrics controller into the new MVC architecture. These tests will include, but not be limited to, testing cases when Github Pull requests exist, and do not exist (for Expertiza and non-Expertiza projects), validation of all data types being stored in the Model / new database tables, appropriate coverage of new and existing graphing helpers, cases when multiple repos and/or pull requests are included in the assignment links, and edge cases when Repos or pull requests were submitted but access rights have not been granted.

==Final Design UML==
We show the UML for the new classes and model we created and implemented in our final project for a overview. Other than these, we also modified some of the existing classes and method, we will give detailed introduction later.

[[File:E2111UML.png]]

==Documentation==
===MetricsHelper===
We have a helper model called MetricsHelper, which contain helper methods for graphing:

====display_github_metrics(parsed_data, authors, dates)====
This function take parsed Github metrics data, authors in a PR, and dates of all commits as parameter. It Creates the bar graph for the Github metrics data. Links the authors with their Github data and assigns them a color. Currently supports up to 6 different colors and will loop if it goes over.

====display_totals_piechart(parsed_data, authors, dates)====
This function take parsed Github metrics data, authors in a PR, and dates of all commits as parameter. It Creates the pie chart for the Github metrics data. Links the authors with their Github data and assigns them a color. It is easy for an instructor to tell the ratio of contribution of each team member with a pie chart.

====chart_options====
This function does not take any parameters. It defines the general settings of the Github metrics chart

====graph_scales====
This function does not take any parameters. It defines the labels and display of the data on the Github metrics chart

===Metric===
This is a new model that saves user's Github email as the identifier of their Github account. With this information, we can connect Github metrics with each students. It also saves the team_id, the participant_id, and the cumulative total commits in the submitted PR for each user. We make this model in a way that it is easy for future team to extend.

====pull_query(hyperlink_data, after)====
This function takes a submitted Github pull request URL and a cursor for pagination in Github GraphQL API as parameter. It Formulate and return the actual query message to send over HTTP request to do the GraphQL query for a Github pull request. For more detailed information, check Github GraphQL API.

====repo_query(hyperlink_data, date, after)====
This function takes a submitted Github repository URL, the date of the project starting date, and a cursor for pagination in Github GraphQL API as parameter. It Formulate and return the actual query message to send over HTTP request to do the GraphQL query for a Github repository. For more detailed information, check Github GraphQL API.

===MetricsController===
The Metrics controller contains all the logics for querying Github metrics and present these metrics with the default show method.

====action_allowed?====
This function does not take any parameters, it checks whether the current user has the required privilege to run certain method.

====create_github_metric(team_id, github_id, total_commits)====
This function takes team_id, github_id, and total_commits as parameter. It does some validations and logic checking of the input value, then it saves all values to database as fields of the Metric model.

====query_assignment_statistics====
This function does not take any parameters, it runs a query against all the link submissions for an entire assignment, populating the DB fields that are used by the view_team in grades heatgrid showing user contributions

====show====
The default show method, which renders the html page that shows all Github metrics with a bar chart and a pie chart.

====authorize_github====
This function redirect the user to Github authorization endpoint to authorize the current user. An authorized user can use the Github API with 5000 rate limits per hour. Unauthorized user only has 60 rate limits per hour, which may not be enough.

====single_submission_initial_query(id)====
This function takes the participant id as parameter. It calls several helper methods to query Github metrics information from links that this team submitted and populate all related instance variable, which will be used in frontend presentation.

====retrieve_github_data====
This function call corresponding method to retrieve Github pull request metrics information or repository information base on different conditions.

====query_all_pull_requests(pull_links)====
This function take all PR URLs that a team submitted and retrieve all Github metrics.

====pull_request_data(hyperlink_data)====
This function takes all hyperlink_data includes pull request number, repository name, owner name of a single PR url as parameter and retrieve all Github metrics for a single PR.

====parse_pull_request_data(github_data)====
This function takes the Github metrics of a PR returned by the query as parameter, parse and store them into corresponding instance variable.

====query_all_merge_statuses====
This function saves each PR's statuses in a hash. This is done through Github REST API not GraphQL.

====retrieve_repository_data(repo_links)====
This function take all repo URLs that a team submitted and retrieve all Github metrics.

====parse_repository_data(github_data)====
This function takes the Github metrics of a repo returned by the query as parameter, parse and store them into corresponding instance variable.

====team_statistics(github_data, data_type)====
This function takes all Github metrics we queried and a data_type that specify the github_data is for a PR or a repo as parameter. It does the basic accounting and store total additions, total deletions, total files changed, total commits, and merge status in instance variable for future presentation purpose.

====count_github_authors_and_dates(author_name, author_email, commit_date)====
This function does the accounting for each author in this PR. It calculates how many commits an author has on each date.

====sort_commit_dates====
This function sort the ruby array that saves the commit information base on date.

====query_commit_statistics(data)====
This function takes the query message as parameter and makes the actual Github api HTTP request with GraphQL and query message.

====query_pull_request_status(pr_object)====
This function takes the global GraphQL id of a PR object as parameter and make the actual Github api HTTP request to query the PR's status info.

===Others===
In addition to the changes mentioned above, we also added a one auth method in auth_controller.rb. We add new show.html.rb page for frontend presentation in app/views/metrics. We also added new testing files.

==Resources==
=====Key Links=====
[https://github.com/smdupor/expertiza/tree/E2111GithubMet E2111 Repository]

[https://github.com/expertiza/expertiza/pull/1917 E2111 Pull Request]

[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2018-_Project_E1858._Github_metrics_integration Original E1858 Documentation]

[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1983._Refactor_E1858,_Github_metrics_integration_view E1983 Revision Documentation]

[https://docs.github.com/en/rest Github API Documentation]

[https://www.youtube.com/watch?v=WX7a4f767QI Demo Video]

CSC/ECE 517 Spring 2021 - E2111. Refactor github metrics integration

2021-03-29T19:52:48Z

Jmhurst: /* Lack of Comments / Opaque Code */

==Abstract==
Github repository and pull request links are submitted as part of many projects in several NC State CSC courses. Instructors would like to compile statistics on these Github links on a per-project, per-team basis, and integrate display of these data within Expertiza (similar to Github Insights). Code was introduced to add this functionality in a [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2018-_Project_E1858._Github_metrics_integration 2018 project, E1858,] which was not merged, and another team re-visited [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1983._Refactor_E1858,_Github_metrics_integration_view this project in 2019], which was not merged. Our team aims to tackle the issues we've found that prevented these past team's code from being merged.

==Functionality==
ADD SECTION INTRODUCING THE FUNCTIONALITY (Description and Maybe one screenshot. Past group went overboard with copy-pasting old screenshots.)

TALK ABOUT OMNIAUTH FOR API AND HOW IT ALL WORKS

==Methods==
The team has synced the original 2018 project code with the current status of the Expertiza/beta branch and begun analyzing the existing code. Prior customer feedback has advised the team to look for coupling, lack of comments/opaque code, and testing issues. We are performing our own end-to-end analysis of the code in addition to this feedback, and have so far found numerous action items. We have separated these action items into Primary and Secondary objectives: Primary objectives are "must-haves" that need to be addressed before the existing code is mergeable. Secondary objectives are "nice-to-haves," or features that were either left out of the original project or would make the integration display more complete.

==Primary Objectives (Immediate Action Items)==
These action items must be addressed before any feature improvements can be attempted.

====API Changes Breaking Functionality====
The Github API has had minor changes since 2018 which are causing the previously working code to either not display, or throw a NoMethodError. We have already analyzed and applied bugfixes to address these minor API changes, but would like to further refine how the API calls are set-up to decouple our code from the API and make it more robust.

ADD CODEBLOCKS TALKING ABOUT THE TWO BUGFIXES

====Coupling to the Grades Controller====
CONTINUED

====Views Contain Large Javascript Blocks====
The views in view.html.erb and view_github_metrics.html.erb in app/views/grades has large JavaScript blocks that were added for the Github metrics functionality. We will move those JavaScript blocks to helper files view_grades_helper.js and view_team_in_grades.js in app/assets/javascipts. The view_team_in_grades.js file is already created with some JavaScript functions that the view_github_metrics.html.erb file already uses, but we will create a new helper file for view.html.erb because it does not currently have any helper files associated with it.

====Methods Out-of-Order====
CHANGE TO CALLING ORDER OR BETTER RE_ORG

====Lack of Comments / Opaque Code====
The majority of methods added in E1858 are lacking comments describing their purpose and functionality and comments were not modified for any modified methods. 
Comments will be added to the following files
<ul>
<li>app/controllers/auth_controller.rb: New methods oauth_login and github_login need comments, existing method action_allowed? needs a comment
<li>app/controllers/grade_controller.rb: Large amount of new methods need comments, existing method "view" needs modification to existing comment
<li>app/helpers/grades_helper.rb: The methods display_github_metrics, chart_options, and graph_scales need comments describing functionality
<li>app/views/assignments/list_submissions.html.erb: New GitHub Metrics column added to table
<li>app/views/grades/_tabbing.html.erb: Class "tmr_tab" will have to be explained
<li>app/views/grades/_teammate_reviews_tab.html.erb: Class "github" will have to be explained
<li>app/views/grades/view.html.erb: JavaScript functionality to be moved to a helper .js file and then commented.
<li>app/views/grades/view_github_metrics.html.erb : JavaScript functionality to be moved to a helper .js file and then commented.
<li>spec/controllers/grades_controller_spec.rb: Comments will have to be added describing the RSpec tests including their purpose and how they achieve their tests
</ul>

====Testing====
SHORT COMMENTARY ON COUPLED TESTING TO GRADES/INSTRUCTOR VIEW. SEE TEST PLAN FOR AT_LENGTH DISCUSSION.

==Secondary Objectives==
The team would like to address one or more of these secondary objectives which were not addressed by prior teams, as long as they do not break existing functionality. These are listed in order of importance.

====Data Permanence (Model)====
The existing code polls Github every time the data is displayed. This makes the rendering code unnecessarily complex and introduces the potential for future data loss. For performance reasons, decoupling, and the benefit of future developers and data permanence, an improvement to this method would be creating a model and database table to hold these data, so that the display is querying the local DB instead of querying out to Github each time the display is rendered. The team would like to extend the proposed new controller to include a new model that will take this data, and only insert the diff into the database. One point of discussion among the team and our mentor will be, should an update query be run each time the page is displayed, or should a burst of queries be run at the expiration of a deadline to populate the database table? Another option may be to create a manual "Github Update" link, but reject all data after a deadline. Yet another option would be to keep all Github data and use the "late data" to call out number of late submissions on the statistics page.

====New Feature: Hide Expertiza Development Team====
At this time, if a team merges in many old commits by the Expertiza Development team, the bar graph statistics are thrown off by including the development team ("Winbobob" et al). If the data permanence objective is met, creating more complex queries should be easier, and the team would like to explore adding a feature to the Github integration display to "hide known contributors," or similar. This would hide Github data from any user included on the "blacklist" which can be manually curated with the short list of Expertiza developers. While this would not address edge cases where an Expertiza developer later participates in a course using Expertiza, this edge case should be quite rare.

==Test Plan==
The 2018 project wrote numerous tests, several of which are deeply coupled into the grades_controller tests. The team proposes several changes to the testing strategy.

====Decouple Tests / Pass Builds====
Builds never passed for this project because, in the current code, a user must be logged in and authorized to Github to render grades#view, and this broke several (expected) test cases. As part of decoupling Github from the grades controller, we expect to decouple and revert tests for the grades controller such that the tests become independent, permitting the build to pass if our Github controller tests pass as expected.

====Detailed Analysis of Github Testing====
The previous team wrote numerous intricate tests for the new functionality. Our team intends to go through these tests line-by-line and ensure that having so many tests is necessary, and the most elegant way to address the issue. We believe that some of the Github refactoring should move methods to become private, which may mean that specific tests for that method shouldn't be used. Alternately, particularly for API authentication which can be tricky, it may be prudent to have discrete tests. Finally, we want to ensure that the tests are actually covering the code -- we are receiving a TravisCI warning about tests without expectations, which needs to be resolved before the code can be merged.

====New Tests for Secondary Objectives====
If a new model is created along with associated database tables, new tests for this model will need to be written first in keeping with TDD principles.

==Resources==
ADD PERTINENT LINKS

==Important Data Structure==
Example of function get_pull_request_details return value(id field in node is omitted for the saving of space):
<nowiki>
{"data"=>{"repository"=>{"pullRequest"=>{"number"=>23281,
"additions"=>183,
"deletions"=>119,
"changedFiles"=>13,
"mergeable"=>"UNKNOWN",
"merged"=>true,
"headRefOid"=>"58f027468c8908160196c34681ff5d224a84d75c",
"commits"=>{"totalCount"=>2,
"pageInfo"=>{"hasNextPage"=>false,
"startCursor"=>"MQ",
"endCursor"=>"Mg"
},
"edges"=>[{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Manish Goregaokar"
},
"additions"=>10, "deletions"=>10,
"changedFiles"=>1,
"committedDate"=>"2019-04-30T02:44:08Z"
}
}
},
{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Maria Sable"},
"additions"=>173,
"deletions"=>109,
"changedFiles"=>12,
"committedDate"=>"2019-04-30T23:54:52Z"
}
}
}
]
}
}
}
}
}</nowiki>

CSC/ECE 517 Spring 2021 - E2111. Refactor github metrics integration

2021-03-29T19:52:23Z

Jmhurst: /* Lack of Comments / Opaque Code */

==Abstract==
Github repository and pull request links are submitted as part of many projects in several NC State CSC courses. Instructors would like to compile statistics on these Github links on a per-project, per-team basis, and integrate display of these data within Expertiza (similar to Github Insights). Code was introduced to add this functionality in a [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2018-_Project_E1858._Github_metrics_integration 2018 project, E1858,] which was not merged, and another team re-visited [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1983._Refactor_E1858,_Github_metrics_integration_view this project in 2019], which was not merged. Our team aims to tackle the issues we've found that prevented these past team's code from being merged.

==Functionality==
ADD SECTION INTRODUCING THE FUNCTIONALITY (Description and Maybe one screenshot. Past group went overboard with copy-pasting old screenshots.)

TALK ABOUT OMNIAUTH FOR API AND HOW IT ALL WORKS

==Methods==
The team has synced the original 2018 project code with the current status of the Expertiza/beta branch and begun analyzing the existing code. Prior customer feedback has advised the team to look for coupling, lack of comments/opaque code, and testing issues. We are performing our own end-to-end analysis of the code in addition to this feedback, and have so far found numerous action items. We have separated these action items into Primary and Secondary objectives: Primary objectives are "must-haves" that need to be addressed before the existing code is mergeable. Secondary objectives are "nice-to-haves," or features that were either left out of the original project or would make the integration display more complete.

==Primary Objectives (Immediate Action Items)==
These action items must be addressed before any feature improvements can be attempted.

====API Changes Breaking Functionality====
The Github API has had minor changes since 2018 which are causing the previously working code to either not display, or throw a NoMethodError. We have already analyzed and applied bugfixes to address these minor API changes, but would like to further refine how the API calls are set-up to decouple our code from the API and make it more robust.

ADD CODEBLOCKS TALKING ABOUT THE TWO BUGFIXES

====Coupling to the Grades Controller====
CONTINUED

====Views Contain Large Javascript Blocks====
The views in view.html.erb and view_github_metrics.html.erb in app/views/grades has large JavaScript blocks that were added for the Github metrics functionality. We will move those JavaScript blocks to helper files view_grades_helper.js and view_team_in_grades.js in app/assets/javascipts. The view_team_in_grades.js file is already created with some JavaScript functions that the view_github_metrics.html.erb file already uses, but we will create a new helper file for view.html.erb because it does not currently have any helper files associated with it.

====Methods Out-of-Order====
CHANGE TO CALLING ORDER OR BETTER RE_ORG

====Lack of Comments / Opaque Code====
The majority of methods added in E1858 are lacking comments describing their purpose and functionality and comments were not modified for any modified methods. 
Comments will be added to the following files
<ul>
<li>app/controllers/auth_controller.rb: New methods oauth_login and github_login need comments, existing method action_allowed? needs a comment
<li>app/controllers/grade_controller.rb: Large amount of new methods need comments, existing method "view" needs modification to existing comment
<li>app/helpers/grades_helper.rb: The methods display_github_metrics, chart_options, and graph_scales need comments describing functionality
<li>app/views/assignments/list_submissions.html.erb: New GitHub Metrics column added to table
<li>app/views/grades/_tabbing.html.erb: Class "tmr_tab" will have to be xplained
<li>app/views/grades/_teammate_reviews_tab.html.erb: Class "github" will have to be explained
<li>app/views/grades/view.html.erb: JavaScript functionality to be moved to a helper .js file and then commented.
<li>app/views/grades/view_github_metrics.html.erb : JavaScript functionality to be moved to a helper .js file and then commented.
<li>spec/controllers/grades_controller_spec.rb: Comments will have to be added describing the RSpec tests including their purpose and how they achieve their tests
</ul>

====Testing====
SHORT COMMENTARY ON COUPLED TESTING TO GRADES/INSTRUCTOR VIEW. SEE TEST PLAN FOR AT_LENGTH DISCUSSION.

==Secondary Objectives==
The team would like to address one or more of these secondary objectives which were not addressed by prior teams, as long as they do not break existing functionality. These are listed in order of importance.

====Data Permanence (Model)====
The existing code polls Github every time the data is displayed. This makes the rendering code unnecessarily complex and introduces the potential for future data loss. For performance reasons, decoupling, and the benefit of future developers and data permanence, an improvement to this method would be creating a model and database table to hold these data, so that the display is querying the local DB instead of querying out to Github each time the display is rendered. The team would like to extend the proposed new controller to include a new model that will take this data, and only insert the diff into the database. One point of discussion among the team and our mentor will be, should an update query be run each time the page is displayed, or should a burst of queries be run at the expiration of a deadline to populate the database table? Another option may be to create a manual "Github Update" link, but reject all data after a deadline. Yet another option would be to keep all Github data and use the "late data" to call out number of late submissions on the statistics page.

====New Feature: Hide Expertiza Development Team====
At this time, if a team merges in many old commits by the Expertiza Development team, the bar graph statistics are thrown off by including the development team ("Winbobob" et al). If the data permanence objective is met, creating more complex queries should be easier, and the team would like to explore adding a feature to the Github integration display to "hide known contributors," or similar. This would hide Github data from any user included on the "blacklist" which can be manually curated with the short list of Expertiza developers. While this would not address edge cases where an Expertiza developer later participates in a course using Expertiza, this edge case should be quite rare.

==Test Plan==
The 2018 project wrote numerous tests, several of which are deeply coupled into the grades_controller tests. The team proposes several changes to the testing strategy.

====Decouple Tests / Pass Builds====
Builds never passed for this project because, in the current code, a user must be logged in and authorized to Github to render grades#view, and this broke several (expected) test cases. As part of decoupling Github from the grades controller, we expect to decouple and revert tests for the grades controller such that the tests become independent, permitting the build to pass if our Github controller tests pass as expected.

====Detailed Analysis of Github Testing====
The previous team wrote numerous intricate tests for the new functionality. Our team intends to go through these tests line-by-line and ensure that having so many tests is necessary, and the most elegant way to address the issue. We believe that some of the Github refactoring should move methods to become private, which may mean that specific tests for that method shouldn't be used. Alternately, particularly for API authentication which can be tricky, it may be prudent to have discrete tests. Finally, we want to ensure that the tests are actually covering the code -- we are receiving a TravisCI warning about tests without expectations, which needs to be resolved before the code can be merged.

====New Tests for Secondary Objectives====
If a new model is created along with associated database tables, new tests for this model will need to be written first in keeping with TDD principles.

==Resources==
ADD PERTINENT LINKS

==Important Data Structure==
Example of function get_pull_request_details return value(id field in node is omitted for the saving of space):
<nowiki>
{"data"=>{"repository"=>{"pullRequest"=>{"number"=>23281,
"additions"=>183,
"deletions"=>119,
"changedFiles"=>13,
"mergeable"=>"UNKNOWN",
"merged"=>true,
"headRefOid"=>"58f027468c8908160196c34681ff5d224a84d75c",
"commits"=>{"totalCount"=>2,
"pageInfo"=>{"hasNextPage"=>false,
"startCursor"=>"MQ",
"endCursor"=>"Mg"
},
"edges"=>[{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Manish Goregaokar"
},
"additions"=>10, "deletions"=>10,
"changedFiles"=>1,
"committedDate"=>"2019-04-30T02:44:08Z"
}
}
},
{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Maria Sable"},
"additions"=>173,
"deletions"=>109,
"changedFiles"=>12,
"committedDate"=>"2019-04-30T23:54:52Z"
}
}
}
]
}
}
}
}
}</nowiki>

CSC/ECE 517 Spring 2021 - E2111. Refactor github metrics integration

2021-03-29T19:35:55Z

Jmhurst: /* Views Contain Large Javascript Blocks */

==Abstract==
Github repository and pull request links are submitted as part of many projects in several NC State CSC courses. Instructors would like to compile statistics on these Github links on a per-project, per-team basis, and integrate display of these data within Expertiza (similar to Github Insights). Code was introduced to add this functionality in a [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2018-_Project_E1858._Github_metrics_integration 2018 project, E1858,] which was not merged, and another team re-visited [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1983._Refactor_E1858,_Github_metrics_integration_view this project in 2019], which was not merged. Our team aims to tackle the issues we've found that prevented these past team's code from being merged.

==Functionality==
ADD SECTION INTRODUCING THE FUNCTIONALITY (Description and Maybe one screenshot. Past group went overboard with copy-pasting old screenshots.)

TALK ABOUT OMNIAUTH FOR API AND HOW IT ALL WORKS

==Methods==
The team has synced the original 2018 project code with the current status of the Expertiza/beta branch and begun analyzing the existing code. Prior customer feedback has advised the team to look for coupling, lack of comments/opaque code, and testing issues. We are performing our own end-to-end analysis of the code in addition to this feedback, and have so far found numerous action items. We have separated these action items into Primary and Secondary objectives: Primary objectives are "must-haves" that need to be addressed before the existing code is mergeable. Secondary objectives are "nice-to-haves," or features that were either left out of the original project or would make the integration display more complete.

==Primary Objectives (Immediate Action Items)==
These action items must be addressed before any feature improvements can be attempted.

====API Changes Breaking Functionality====
The Github API has had minor changes since 2018 which are causing the previously working code to either not display, or throw a NoMethodError. We have already analyzed and applied bugfixes to address these minor API changes, but would like to further refine how the API calls are set-up to decouple our code from the API and make it more robust.

ADD CODEBLOCKS TALKING ABOUT THE TWO BUGFIXES

====Coupling to the Grades Controller====
CONTINUED

====Views Contain Large Javascript Blocks====
The views in view.html.erb and view_github_metrics.html.erb in app/views/grades has large JavaScript blocks that were added for the Github metrics functionality. We will move those JavaScript blocks to helper files view_grades_helper.js and view_team_in_grades.js in app/assets/javascipts. The view_team_in_grades.js file is already created with some JavaScript functions that the view_github_metrics.html.erb file already uses, but we will create a new helper file for view.html.erb because it does not currently have any helper files associated with it.

====Methods Out-of-Order====
CHANGE TO CALLING ORDER OR BETTER RE_ORG

====Lack of Comments / Opaque Code====
CONTINUED

====Testing====
SHORT COMMENTARY ON COUPLED TESTING TO GRADES/INSTRUCTOR VIEW. SEE TEST PLAN FOR AT_LENGTH DISCUSSION.

==Secondary Objectives==
The team would like to address one or more of these secondary objectives which were not addressed by prior teams, as long as they do not break existing functionality. These are listed in order of importance.

====Data Permanence (Model)====
The existing code polls Github every time the data is displayed. This makes the rendering code unnecessarily complex and introduces the potential for future data loss. For performance reasons, decoupling, and the benefit of future developers and data permanence, an improvement to this method would be creating a model and database table to hold these data, so that the display is querying the local DB instead of querying out to Github each time the display is rendered. The team would like to extend the proposed new controller to include a new model that will take this data, and only insert the diff into the database. One point of discussion among the team and our mentor will be, should an update query be run each time the page is displayed, or should a burst of queries be run at the expiration of a deadline to populate the database table? Another option may be to create a manual "Github Update" link, but reject all data after a deadline. Yet another option would be to keep all Github data and use the "late data" to call out number of late submissions on the statistics page.

====New Feature: Hide Expertiza Development Team====
At this time, if a team merges in many old commits by the Expertiza Development team, the bar graph statistics are thrown off by including the development team ("Winbobob" et al). If the data permanence objective is met, creating more complex queries should be easier, and the team would like to explore adding a feature to the Github integration display to "hide known contributors," or similar. This would hide Github data from any user included on the "blacklist" which can be manually curated with the short list of Expertiza developers. While this would not address edge cases where an Expertiza developer later participates in a course using Expertiza, this edge case should be quite rare.

==Test Plan==
The 2018 project wrote numerous tests, several of which are deeply coupled into the grades_controller tests. The team proposes several changes to the testing strategy.

====Decouple Tests / Pass Builds====
Builds never passed for this project because, in the current code, a user must be logged in and authorized to Github to render grades#view, and this broke several (expected) test cases. As part of decoupling Github from the grades controller, we expect to decouple and revert tests for the grades controller such that the tests become independent, permitting the build to pass if our Github controller tests pass as expected.

====Detailed Analysis of Github Testing====
The previous team wrote numerous intricate tests for the new functionality. Our team intends to go through these tests line-by-line and ensure that having so many tests is necessary, and the most elegant way to address the issue. We believe that some of the Github refactoring should move methods to become private, which may mean that specific tests for that method shouldn't be used. Alternately, particularly for API authentication which can be tricky, it may be prudent to have discrete tests. Finally, we want to ensure that the tests are actually covering the code -- we are receiving a TravisCI warning about tests without expectations, which needs to be resolved before the code can be merged.

====New Tests for Secondary Objectives====
If a new model is created along with associated database tables, new tests for this model will need to be written first in keeping with TDD principles.

==Resources==
ADD PERTINENT LINKS

==Important Data Structure==
Example of function get_pull_request_details return value(id field in node is omitted for the saving of space):
<nowiki>
{"data"=>{"repository"=>{"pullRequest"=>{"number"=>23281,
"additions"=>183,
"deletions"=>119,
"changedFiles"=>13,
"mergeable"=>"UNKNOWN",
"merged"=>true,
"headRefOid"=>"58f027468c8908160196c34681ff5d224a84d75c",
"commits"=>{"totalCount"=>2,
"pageInfo"=>{"hasNextPage"=>false,
"startCursor"=>"MQ",
"endCursor"=>"Mg"
},
"edges"=>[{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Manish Goregaokar"
},
"additions"=>10, "deletions"=>10,
"changedFiles"=>1,
"committedDate"=>"2019-04-30T02:44:08Z"
}
}
},
{"node"=>{"id"=>"MDE3OlB1b...",
"commit"=>{"author"=>{"name"=>"Maria Sable"},
"additions"=>173,
"deletions"=>109,
"changedFiles"=>12,
"committedDate"=>"2019-04-30T23:54:52Z"
}
}
}
]
}
}
}
}
}</nowiki>

CSC/ECE 517 Spring 2021 - E2100. Tagging report for students

2021-03-25T14:37:42Z

Jmhurst: Added Important Links section

This page details project documentation for the Spring 2021, "E2100 Tagging report for students" project, which aims to assist students with finding, and completing, incomplete "review tags" on an assignment using a dynamically generated heatgrid.

==Introduction to Expertiza==
[http://http://expertiza.ncsu.edu/ Expertiza] is an [http://en.wikipedia.org/wiki/Open-source_software Open Source] [http://rubyonrails.org/ Rails] application which is used by instructors and students for creating assignments and submitting peer reviews. Expertiza allows the instructor to create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. The Source code of the application can be cloned from [https://github.com/expertiza/expertiza/ Github].

==About Review Tagging==
Review "tags" are a form of feedback on Expertiza where students "tag" (classify) text from peer reviews, using Yes or No answers to questions chosen specifically for each tag deployment. The parameters can include helpfulness, positivity, suggestions, whether a review offered mitigation, and other parameters depending on the Answer Tag Deployment, and the number of tags per deployment is arbitrary. These labeled data are then made available to Expertiza researchers for use in developing Natural Language Processing (NLP) / Machine Learning (ML) algorithms. Tagging is only collected for reviews where the reviewer wrote enough words to be useful as labeled data for NLP research.

[[File:tagDeploymentExample.png|700px]]

==Problem Statement==
It can be difficult for students to find a tag they missed on the Team View page, and other teams are working with ML algorithms to "pre-tag" as many reviews as possible, leading to a granular field of completed/incomplete tags. For example, an assignment with two rounds of reviewing, ten questions per review, twelve reviews, and a 5-parameter tag deployment could contain as few as zero or as many as one thousand, two hundred tag prompts for a single student to complete.

At this time, the only tagging feedback students see are the tag prompts and a Javascript counter with a numeric representation of how many tags have not been completed. In order to find a missed tag, students have to scroll the page and manually search for tags that aren't done.

In order to help students complete all the tags for an assignment, we propose a new, dynamically generated heatgrid on the "Your Scores" view that breaks down reviews and tags by round, question, and review, which uses visual cues to help students find incomplete tags. The heatgrid shows both a total count of "complete" out of "available" tags, and individual tags with color-scaled feedback for completeness.

'''The new heatgrid must handle these requirements, and display visual feedback accordingly:'''
* Single and multiple round reviews
* Reviews with and without tag prompts
* Dynamically update as tags are completed
* Different tag deployments will use different numbers of tags per review
* Use of existing review row-sort functionality
* Not rely on the database for tag data, as AnswerTag database rows may not exist until tags are clicked.
* Give useable feedback for all users (characters as well as color-coding)
* Only be shown when a tags have been deployed to an assignment
* Show the total progress of tagging in the format, "249 out of 315"

==Implementation==
Our proposed solution is a visual feedback aid, primarily for student users. Tagging data, when input, is stored to the database dynamically using jQuery. For these reasons, and to facilitate dynamic updates, we have chosen to implement this functionality entirely on the client-side of the application using Javascript and jQuery.

===Design Patterns===
The feedback aid consists of a new heat grid using an HTML table, dynamically generated client-side with Javascript, that uses jQuery to extract the requisite data from the page. The design of this functionality utilizes ''Adapter'' and ''Singleton'' design patterns. ''Singleton'' is present in this design because there shall only ever exist a single tagging heatgrid in the application. While the heatgrid contains an aggregation of other objects, there shall not be multiple heatgrids for multiple rounds or reviews competing for space on the page. ''Adapter'' is used in building the code which adapts from the existing backend, where Tag objects are not created until they are populated by the NLP algorithm or clicked by a student, to the heatgrid which must be able to find and display all questions and tags, regardless of their status or backend implementation.

===Flow Diagram===
The presence or absence of tag prompts are detected at page load, and the heat grid is rendered as appropriate. The flowchart below details the new Javascript functions which were implemented to accomplish this goal, and the associated program flow.

'''Your Scores Implementation Flow Diagram'''

[[File:View_teamRevA.png]]

'''Heat grid for a two-round review'''

[[File:HeatGridDemo.png|1100px]]

===Code Style===
A note on code style: Because this project utilizes so much Javascript, we tried to keep the code style consistent and true to best practices for both Ruby and Javascript, depending on which language we were working in. Therefore, any code or variables from the Ruby section should use snake_case, and code in the Javascript section should use lowerCamelCase.

===Server-Side Implementation: "Your Scores" and "Alternate View"===
Because most of the logic is implemented on the client-side, the server-side implementation is fairly simple.

===="Your Scores" view (/views/grades/view_team.html.erb)====
We added an empty HTML <<nowiki />table> tag with id and tag information that is used to generate the heatgrid with Javascript.

[[File:view_team_table_tag.png]]

In order to simplify the jQuery which parses the rendered page for tags, we added three numeric and one boolean jQueryable data- fields and a new id field to the td container tag which holds each review text (and the associated tags, if present). We used existing Rails variables to populate these data and id fields. The id is set using "rr" for reviewrow (shortened for brevity) plus the question and review number (j and index). We also created fields for data-round (round number) data-question_num (question number), data-review_num (review number), and data-has_tag (boolean whether tags exist).

Style Note: the "data-" is the key queried by jQuery, so a dash is used. The string after the dash is free for us to choose the style, so we used Ruby style to match the convention on the rest of the application.

[[File:tdRailsTag.png]]

Finally, we added a call inside the <code>document.Ready()</code> event handler to tagActionOnLoad(), which starts the rendering code for the client-side Javascript once the page has loaded and tags (if available) are present for counting. We also added a jQuery of our new heat grid table within the onClick for the "Hide Tags" link to ensure our heatgrid is hidden when tag prompts are hidden.

[[File:view_team_javascript.png]]

===="Alternate View" (/views/grades/view_my_scores.html.erb)====

We added function calls to countTotalTags() and countTaggedTags() inside the document.Ready() event handler to calculate the total number of tags and the total number of tagged tags on the page that currently being rendered. We also added div HTML element with id tag_stats and render this element with string "Tag Finished: tagged_tags/total_tags". For example, if there are 370 tags in total on this page and 10 of them has already been tagged, then 10/370 will be displayed. This element will also be updated dynamically when users make tag changes on this page.

[[File:alternate_view_html_code.png|1150px]]

===Client-Side Implementation: "Your Scores" (view_team.html.erb)===
The majority of the logic is implemented on the client side. After the page is loaded and the rest of the ReactJS interface is rendered, a dynamically rendered heatgrid is rendered on the "Your Scores" view. A series of functions included in Javascript asset files <code>view_team_in_grades.js</code> and <code>answer_tags.js</code> are called which render the appropriate element for each view. These functions are described below, organized in calling order.

====Initialization and Rendering - view_team_in_grades.js====
=====tagActionOnLoad()=====
Called by <code>document.Ready()</code> event handler, this function first calls <code><nowiki> getTagPrompts()</nowiki></code>, determines if zero exist, and exits if so. If tag prompts exist, it continues to call other functions which query the page for information and render the tag heatgrid.
[[File:tagActionOnLoad.png]]

=====getTagPrompts()=====
A simple Javascript query that collects all the tag prompts as an HTMLCollection for use with other function.

[[File:getTagPrompts.png]]

=====calcTagRatio()=====
A mathematics helper function that uses the tag prompt collection to gather data used in populating the tag heatgrid header ("tag fraction") which reads, for example, "25 out of 350". This function gathers all the numeric data, as well as calculating the completion ratio as a decimal and adapting this decimal to a value that can be used to reference heatgrid color classes (c0 grey, c1 red, c2 orange, c3 yellow, c4 light green, c5 dark green). Note that we avoided using the light green class because it's too easily confused with dark green -- meaning "all done" -- so logic in this function ensures we skip straight from c3 yellow @ .9999 completion, to c5 dark green at 1.0000 completion.

[[File:calcTagRatio.png]]

=====getRowData()=====
This method uses jQuery to retrieve the row containers for all questions in all rounds of this review, whether they contain tag prompts or not. This allows us to populate the heatgrid with 'grey - universal No-symbol' for reviews whose length metric is too low to have a tag prompt associated.

[[File:getRowData.png]]

=====countRounds()=====
This function parses the rows of the review to determine how many rounds of reviews exist in the assignment. This allows us to intelligently print "Round 2 -- Question 3" for multi round reviews, or "Question 3" for single round reviews.

[[File:countRounds.png]]

=====getGridWidth()=====
This function parses the row data to determine how many tag prompts were used in this deployment using a simple selection search for the largest number of tags in a review. Since different tag deployments use different numbers of prompts, the rendering of the heatgrid needs to address this flexibility. Furthermore, the design of this function will allow for future flexibility of tag deployments, i.e., where different reviews in the same assignment or round may have different numbers of tags.

[[File:getGridWidth.png]]

=====drawTagGrid()=====
This is the master function for the rendering of the tag grid. This function sets up tooltip text, calls countRounds() and getGridWidth() and passes control to the three sub-functions, drawHeader(), drawQuestionRow(), and drawReviewRow(), depending on what type of item needs to be rendered next.

[[File:drawTagGrid.png]]

=====drawHeader()=====
This function draws the two-row header of the tag heatgrid which will contain "Tags Completed" and "15 out of 425". It also calls addToolTip() to add the tooltips for expand/contract, and the color legend for the tag fraction row.

[[File:drawHeader.png]]

=====drawQuestionRow()=====
This function is called once per question during the rendering process, and draws the row which will read, "Question 3" or "Round 2 -- Question 3". This also calls addToolTip() to add a color legend tooltip for the body of the heatgrid.

[[File:drawQuestionRow.png]]

=====drawReviewRow()=====
This function draws each row containing the presence, or absence of, tag prompts for each review. These contain R.# (Review.number), and a unicode symbol (set in the global variables). The unicode symbol is to enable accessibility for users who cannot see the red/green spectrum clearly, and will be populated with either a Universal No to mean no tags available, a Warning symbol, to mean tags not done, or a Check-Mark to indicate tags done.

[[File:drawReviewRow.png]]

Lastly, a call is made to updateTagsFraction() as part of the initial render to update the "12 out of 230" header row in the tag grid.

====Updating and Interaction Functions - view_team_in_grades.js====
=====tagActionOnUpdate()=====
Each time a tag prompt is changed, this function is called to update the heatgrid. This function calls several other functions, some of which have been used previously it the rendering step.

[[File:tagActionOnUpdate.png]]

=====updateTagsFraction()=====
This function is used both once in the rendering step, and again each time a tag prompt is changed. This function takes the data from calcTagRatio() and updates the contents and color of the header row in the tag heatgrid ("12 out of 220")

[[File:updateTagsFraction.png]]

=====updateTagGrid()=====
This function queries the tag prompts on the page, and uses these data to update the tag heatgrid class and text.

[[File:updateTagGrid.png]]

=====toggleHeatGridRows()=====
This function "Collapses" the heatgrid down to only the two header rows. Note that this function is called both onClick of the heatgrid header, so a user can collapse the grid manually, and automatically when the tag fraction is updated. This ensures that a user visiting the page with all tags complete will see a collapsed heatgrid by default.

[[File:toggleHeatGridRows.png]]

====Stylesheets - grades.scss====
We augmented the stylesheets used in grades so that the new heatgrid could be styled as small as possible to save page real estate. To do so, we added new classes in grades.scss

[[File:stylesheetsGrades_scss.png]]

===Client-Side Implementation: "Alternate View" (view_my_scores.html.erb)===
=====countTotalTags()=====
A simple JS function to count all the tag elements on current page. It will be invoked when the page first being rendered and also when any tags are changed by users.
[[File:countTotalTags.png]]

=====countTaggedTags()=====
A simple JS function to count the number of tagged tags on the current page. It will be invoked when the page first being rendered and when any tags are changed by users.
[[File:countTaggedTags.png]]

=====Additional Logic Check=====
This JS helper dynamically render the page when any tags are changed by users. Since we adopt different logic on two view pages, we added a condition check to tell which page we are on and apply the corresponding logic.
[[File:answer_tag.png]]

==Test Plan==

===Functional Testing===
Since we did not make any changes to the models, controllers, or the database backend of Expertiza, we did construct any new RSpec tests for any model or controller behavior.

We did perform functional testing using Capybara, to ensure that our changes to the Embedded Ruby views work and are preserved in future modifications of the code.

Our new functional test was placed within spec/features. The file is named view_team_spec.rb and its purpose is to ensure that the table for the heatgrid is generated when accessing the "Your Scores" section. We built this new test using some code from pre-existing feature tests, namely peer_review_spec.rb, which tests the functionality of user access and leaving a review for another project.

You can run our created feature test with the following command in the expertiza directory. 
''rspec spec/features/view_team_spec.rb''

Expected Output:
1 example, 0 failures

====Capybara Test Code====
The following section of code sets up the tests with factory methods located in spec/factories/factories.rb. It creates a User First and Second who are located on Team 1, a User Third who is located on Team 2. It also maps the third user to review team 1 and the first and second users to review team 2.
''before(:each) do
create(:assignment, name: "TestAssignment", directory_path: 'test_assignment')
create_list(:participant, 3)
create(:assignment_node)
create(:deadline_type, name: "submission")
create(:deadline_type, name: "review")
create(:deadline_type, name: "metareview")
create(:deadline_type, name: "drop_topic")
create(:deadline_type, name: "signup")
create(:deadline_type, name: "team_formation")
create(:deadline_right)
create(:deadline_right, name: 'Late')
create(:deadline_right, name: 'OK')
create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now.in_time_zone + 1.day)
create(:topic)
create(:topic, topic_name: "TestReview")
create(:team_user, user: User.where(role_id: 1).first)
create(:team_user, user: User.where(role_id: 1).second)
create(:assignment_team)
create(:team_user, user: User.where(role_id: 1).third, team: AssignmentTeam.second)
create(:signed_up_team)
create(:signed_up_team, team_id: 2, topic: SignUpTopic.second)
create(:assignment_questionnaire)
create(:question)
create(:submission_record)
create(:submission_record, team_id: 2)
create(:review_response_map, reviewer_id: User.where(role_id: 1).third.id)
create(:review_response_map, reviewer_id: User.where(role_id: 1).first.id, reviewee: AssignmentTeam.second)
create(:review_response_map, reviewer_id: User.where(role_id: 1).second.id, reviewee: AssignmentTeam.second)
create(:review_grade, review_graded_at: Time.now.in_time_zone)
end''

The following section of code executes the common functionality used in the test of logging in as the third user and then navigating to their "Your Scores" page.
''def load_your_scores
login_as(User.where(role_id:1).third.name)
expect(page).to have_content "User: " + User.where(role_id:1).third.name
click_link "Assignments"
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Your scores"
expect(page).to have_content "Alternate View"
click_link "Your scores"
expect(page).to have_content 'Summary Report for assignment: TestAssignment'
end''

The next section of code is the add_review method, which logs in as the first user and adds a review to team 2 which the third user is a part of. This section of code was heavy influenced by "peer_review_spec.rb"
''def add_review
login_as(User.where(role_id:1).first.name)
expect(page).to have_content "User: " + User.where(role_id:1).first.name
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Others' work"
click_link "Others' work"
expect(page).to have_content 'Reviews for "TestAssignment"'
choose "topic_id"
click_button "Request a new submission to review"
click_link "Begin"
fill_in "responses[0][comment]", with: "HelloWorld"
select 3, from: "responses[0][score]"
click_button "Submit Review"
expect(page).to have_content "Your response was successfully saved."
click_link "Logout"
end''

The final section of our test is the actual feature test that checks the heatmap. This section of the code utilizes both of the above defined "load_your_scores" and "add_review" methods to accomplish the task of viewing the heatmap for the third User.
''it "Should contain html target and javascript calls for tag heatgrid" do
# Load Summary Report with no reviews
load_your_scores
expect(page).to have_content "Average peer review score: "
# Add review as first user
click_link "Logout"
add_review
# View Your Scores with one review
load_your_scores
# Check for target to build new heatgrid onto
expect(page.body).to include <nowiki>'<table id="tag_heat_grid" class="tag_heat_grid"></table>'</nowiki>
# Check for Javascript action to generate the heatgrid
expect(page.body).to include 'tagActionOnLoad();'
# Check for Javascript action to turn the heatgrid on and off with answer tag toggle
expect(page.body).to include "$('.tag_heat_grid').toggle();"
end''

===Acceptance Testing===
For manual testing purposes, we logged in as instructor6 in order to:
* View the tagging report as an instructor/TA (Instructor tagging is permitted, and tags are separate from Student Tags) on '''view_team'''
* Impersonate a student to view the tagging report on '''view_team''' as a student
* View the mini-report on '''Alternate View''' both as students and instructors
* Ensure that changing of tags updates the Tagging Report
* Ensure that the Tagging Report interacts well with the existing layout
* Ensure that the Tagging Report is hidden/shown when "Hide Tags" is clicked
* Ensure that the Tagging Report accordions as expected when clicked
* Confirm that mouseover events display the expected tooltips, including color scheme for the "10 out of 250" cell as well as color scheme for the main report body, and the Expand/Collapse tooltip
* Ensure that the Tagging Report is hidden when there are no Tags available for a '''view_team''' page
* Ensure that the Tagging Report was compatible with different tag deployments containing different numbers of tags

 
A good example of functionality for a project with reviews and one with reviews can both be found with student7856. Program 2 has plenty of reviews that can be tagged and untagged and checked that the heatmap functionality is working. Program 1 does not have any reviews to tag and was checked to ensure that nothing broke due to not having any reviews to tag.

====Manual Acceptance Testing Instructions====
1. Login to Expertiza using [http://expertiza-tuffy-354839034.us-east-2.elb.amazonaws.com/ production link hosted on Amazon Web Services]. Use user id: '''instructor6''' with password: '''password'''

2. Navigate to an assignment that has a tag deployment. Many recent assignments (2017-present) which have reviews have tag deployments. Click "Assignments", then the submissions icon. 
[[File:clickNav.PNG|1000px]] 

3. Both instructors and students can enter tag data. To view the instructor view, tags, and tagging heatgrid, click "Assign Grade". To impersonate a student, click the Team Member name link. 
[[File:clickViews.PNG|1000px]] 

4. After impersonating a student, select an assignment. We suggest Program 2 or OSS Projects from a CSC/ECE517 section. 
[[File:clickStudentView.PNG|1000px]] 

5. Click "Your Scores" to view the main View Team page which includes most of the functionality, including the tag heatgrid. 
[[File:clickYourScores.PNG|1000px]] 

'''6. Items to Check:''' 
a. On the main View Team Page, click a question row and change one or more tag prompts, to ensure the heatgrid and fraction are updated. 
b. Mouseover the fraction and body of the heatgrid to view the two tooltips describing color scheme. 
c. Click "Hide tags/Show Tags" to show and hide all tag-related data. 
d. Click "Toggle Question List" to ensure the layout is working consistently and as expected. 
e. Click the row-sorter for question or average, and change one or more tag prompts to ensure the heatgrid is still updating the correct tags 
[[File:clickSpots.PNG|1000px]] 

7. To view the "Alternate View", navigate back and click "Alternate View". This view should show a small counter near the top of the page with tag completion data. 
You should see the following page. 
[[File:taggingAlternateView.PNG|1000px]] 

8. To view a different assignment, if you have impersonated a student, in addition to using Browser-Back, you may have to click "Revert" in the impersonate dialogue. 
9. We suggest now impersonating '''student7856''' and navigating to '''Program 2''', then '''Your Scores''' to view a series of reviews with all tags completed so you can see how the interface behaves. You will find that the tag grid is collapsed at document load when all tags are completed so it is out of the way, and can be expanded by clicking the header. 

10. Finally, we suggest viewing an assignment which has reviews, but does not have a tag deployment assigned, to ensure none of the heatgrid materials related to tagging are displayed. We suggest navigating to this assignment from 2016 to test this functionality: 
[[File:taglessAssignment.PNG|1000px]] 

==Additional Modifications -- Bugfix for Average Review Score on View Team==
We discovered another bug where assignments with multiple rounds of reviewing did not display the average review score for the assignment. These are the screenshots of this bug before and after a patch was applied:
===Screenshots===
'''Unpatched Bug: ''' [[File:avgScoreBugUnpatched.png|1000px]]

'''Patch Applied: ''' [[File:avgScoreBugPatched.png|1000px]]

===Bugfix Description===
We discovered that this bug was due to a change in how scores are calculated, averaged, and stored. Average scores are computed one round of reviews at a time, and for single-round reviews, retrieved using <code>@pscore[:review]</code>. For multi-round reviews, the scores are stored as <code>@pscore[:review1]</code> and <code>@pscore[:review2]</code>. The existing code in view_team.html.erb was only checking for <code>@pscore[:review]</code>, and printing "There are no reviews for this assignment" if the key, <code>:review</code> did not have data associated. We added a snippet to test for <code>@pscore[:review1]</code> and <code>@pscore[:review2]</code>, and compute and print the average score across both reviews accordingly.

The team acknowledges that this bugfix is not ideal in implementation, but is an effective short-term fix. A more elegant solution will require significant modifications to how review score models are handled in the backend, which will affect other areas of the application, and needs to be planned more carefully than a one-shot quick fix.

The changes we made to this code are:

[[File:codeAvgScoreBugPatched.png]]

==Important Links==
E2100 Github repo - https://github.com/smdupor/expertiza 
E2100 Pull Request - https://github.com/expertiza/expertiza/pull/1895 
Files Changed - https://github.com/expertiza/expertiza/pull/1895/files

CSC/ECE 517 Spring 2021 - E2100. Tagging report for students

2021-03-19T21:01:25Z

Jmhurst: /* Testing */

This page details project documentation for the Spring 2021, "E2100 Tagging report for students" project, which aims to assist students with finding, and completing, incomplete "review tags" on an assignment using a dynamically generated heatgrid.

==Introduction to Expertiza==
[http://http://expertiza.ncsu.edu/ Expertiza] is an [http://en.wikipedia.org/wiki/Open-source_software Open Source] [http://rubyonrails.org/ Rails] application which is used by instructors and students for creating assignments and submitting peer reviews. Expertiza allows the instructor to create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. The Source code of the application can be cloned from [https://github.com/expertiza/expertiza/ Github].

==About Review Tagging==
Review "tags" are a form of feedback on Expertiza where students "tag" (classify) text from peer reviews based on parameters specific to each tag deployment. The parameters can include helpfulness, positivity, suggestions, whether a review offered mitigation, and other parameters depending on the Answer Tag Deployment. These labeled data are then made available to Expertiza researchers for use in developing Natural Language Processing (NLP) / Machine Learning (ML) algorithms. Tagging is only collected for reviews where the text is a sufficient length to be useful as labeled data for NLP research.

==Problem Statement==
It can be difficult for students to find a tag they missed on the Team View page, and other teams are working with ML algorithms to "pre-tag" as many reviews as possible, leading to a granular field of completed/incomplete tags. For example, an assignment with two rounds of reviewing, ten questions per review, twelve reviews, and a 5-parameter tag deployment could contain as few as zero or as many as one thousand, two hundred tag prompts for a single student to complete.

At this time, the only tagging feedback students see are the tag prompts and a Javascript counter with a numeric representation of how many tags have not been completed. In order to find a missed tag, students have to scroll the page and manually search for tags that aren't done.

In order to help students complete all the tags for an assignment, we propose a new, dynamically generated heatgrid on the "Your Scores" view that breaks down reviews and tags by round, question, and review, which uses visual cues to help students find incomplete tags. The heatgrid shows both a total count of "complete" out of "available" tags, and individual tags with color-scaled feedback for completeness.

'''The new heatgrid must handle these requirements, and display visual feedback accordingly:'''
* Single and multiple round reviews
* Reviews with and without tag prompts
* Dynamically update as tags are completed
* Different tag deployments will use different numbers of tags per review
* Use of existing review row-sort functionality
* Not rely on the database for tag data, as AnswerTag database rows may not exist until tags are clicked.
* Give useable feedback for all users (characters as well as color-coding)
* Only be shown when a tags have been deployed to an assignment
* Show the total progress of tagging in the format, "249 out of 315"

==Implementation==
Our proposed solution is a visual feedback aid which exists purely for student users. Tagging data, when input, is stored to the database dynamically using jQuery. For these reasons, and to facilitate dynamic updates, we have chosen to implement this functionality entirely on the client-side of the application using Javascript and jQuery. The presence or absence of tag prompts are detected at page load, and the heat grid is rendered as appropriate. The flowchart below details the new Javascript functions which were implemented to accomplish this goal, and the associated program flow.

'''Your Scores Flow Diagram'''

[[File:View_teamRevA.png]]

'''Heat grid for a two-round review'''

[[File:HeatGridDemo.png|1100px]]

===Code Style===
A note on code style: Because this project utilizes so much Javascript, we tried to keep the code style consistent and true to best practices for both Ruby and Javascript, depending on which language we were working in. Therefore, any code or variables from the Ruby section should use snake_case, and code in the Javascript section should use lowerCamelCase.

===Server-Side Implementation: "Your Scores" and "Alternate View"===
Because most of the logic is implemented on the client-side, the server-side implementation is fairly simple.

===="Your Scores" view (/views/grades/view_team.html.erb)====
We added an empty HTML <<nowiki />table> tag with id and tag information that is used to generate the heatgrid with Javascript.

[[File:view_team_table_tag.png]]

In order to simplify the jQuery which parses the rendered page for tags, we added three numeric and one boolean jQueryable data- fields and a new id field to the td container tag which holds each review text (and the associated tags, if present). We used existing Rails variables to populate these data and id fields. The id is set using "rr" for reviewrow (shortened for brevity) plus the question and review number (j and index). We also created fields for data-round (round number) data-question_num (question number), data-review_num (review number), and data-has_tag (boolean whether tags exist).

A word on style choices: the "data-" is the key queried by jQuery, so a dash is used. The string after the dash is free for us to choose the style, so we used Ruby style to match the convention on the rest of the application.

[[File:tdRailsTag.png]]

Finally, we added a call inside the <code>document.Ready()</code> event handler to tagActionOnLoad(), which starts the rendering code for the client-side Javascript once the page has loaded and tags (if available) are present for counting. We also added a jQuery of our new heat grid table within the onClick for the "Hide Tags" link to ensure our heatgrid is hidden when tag prompts are hidden.

[[File:view_team_javascript.png]]

===="Alternate View" (/views/grades/view_my_scores.html.erb)====

We added function calls to countTotalTags() and countTaggedTags() inside the document.Ready() event handler to calculate the total number of tags and the total number of tagged tags on the page that currently being rendered. We also added div HTML element with id tag_stats and render this element with string "Tag Finished: tagged_tags/total_tags". For example, if there are 370 tags in total on this page and 10 of them has already been tagged, then 10/370 will be displayed. This element will also be updated dynamically when users make tag changes on this page.

[[File:alternate_view_html_code.png]]

===Client-Side Implementation: "Your Scores" (view_team.html.erb)===
The majority of the logic is implemented on the client side. After the page is loaded and the rest of the ReactJS interface is rendered, a dynamically rendered heatgrid is rendered on the "Your Scores" view. A series of functions included in Javascript asset files <code>view_team_in_grades.js</code> and <code>answer_tags.js</code> are called which render the appropriate element for each view. These functions are described below, organized in calling order.

====Initialization and Rendering - view_team_in_grades.js====
=====tagActionOnLoad()=====
Called by <code>document.Ready()</code> event handler, this function first calls <code><nowiki> getTagPrompts()</nowiki></code>, determines if zero exist, and exits if so. If tag prompts exist, it continues to call other functions which query the page for information and render the tag heatgrid.
[[File:tagActionOnLoad.png]]

=====getTagPrompts()=====
A simple Javascript query that collects all the tag prompts as an HTMLCollection for use with other function.

[[File:getTagPrompts.png]]

=====calcTagRatio()=====
A mathematics helper function that uses the tag prompt collection to gather data used in populating the tag heatgrid header ("tag fraction") which reads, for example, "25 out of 350". This function gathers all the numeric data, as well as calculating the completion ratio as a decimal and adapting this decimal to a value that can be used to reference heatgrid color classes (c0 grey, c1 red, c2 orange, c3 yellow, c4 light green, c5 dark green). Note that we avoided using the light green class because it's too easily confused with dark green -- meaning "all done" -- so logic in this function ensures we skip straight from c3 yellow @ .9999 completion, to c5 dark green at 1.0000 completion.

[[File:calcTagRatio.png]]

=====getRowData()=====
This method uses jQuery to retrieve the row containers for all questions in all rounds of this review, whether they contain tag prompts or not. This allows us to populate the heatgrid with 'grey - universal No-symbol' for reviews whose length metric is too low to have a tag prompt associated.

[[File:getRowData.png]]

=====countRounds()=====
This function parses the rows of the review to determine how many rounds of reviews exist in the assignment. This allows us to intelligently print "Round 2 -- Question 3" for multi round reviews, or "Question 3" for single round reviews.

[[File:countRounds.png]]

=====getGridWidth()=====
This function parses the row data to determine how many tag prompts were used in this deployment using a simple selection search for the largest number of tags in a review. Since different tag deployments use different numbers of prompts, the rendering of the heatgrid needs to address this flexibility. Furthermore, the design of this function will allow for future flexibility of tag deployments, i.e., where different reviews in the same assignment or round may have different numbers of tags.

[[File:getGridWidth.png]]

=====drawTagGrid()=====
This is the master function for the rendering of the tag grid. This function sets up tooltip text, calls countRounds() and getGridWidth() and passes control to the three sub-functions, drawHeader(), drawQuestionRow(), and drawReviewRow(), depending on what type of item needs to be rendered next.

[[File:drawTagGrid.png]]

=====drawHeader()=====
This function draws the two-row header of the tag heatgrid which will contain "Tags Completed" and "15 out of 425". It also calls addToolTip() to add the tooltips for expand/contract, and the color legend for the tag fraction row.

[[File:drawHeader.png]]

=====drawQuestionRow()=====
This function is called once per question during the rendering process, and draws the row which will read, "Question 3" or "Round 2 -- Question 3". This also calls addToolTip() to add a color legend tooltip for the body of the heatgrid.

[[File:drawQuestionRow.png]]

=====drawReviewRow()=====
This function draws each row containing the presence, or absence of, tag prompts for each review. These contain R.# (Review.number), and a unicode symbol (set in the global variables). The unicode symbol is to enable accessibility for users who cannot see the red/green spectrum clearly, and will be populated with either a Universal No to mean no tags available, a Warning symbol, to mean tags not done, or a Check-Mark to indicate tags done.

[[File:drawReviewRow.png]]

Lastly, a call is made to updateTagsFraction() as part of the initial render to update the "12 out of 230" header row in the tag grid.

====Updating and Interaction Functions - view_team_in_grades.js====
=====tagActionOnUpdate()=====
Each time a tag prompt is changed, this function is called to update the heatgrid. This function calls several other functions, some of which have been used previously it the rendering step.

[[File:tagActionOnUpdate.png]]

=====updateTagsFraction()=====
This function is used both once in the rendering step, and again each time a tag prompt is changed. This function takes the data from calcTagRatio() and updates the contents and color of the header row in the tag heatgrid ("12 out of 220")

[[File:updateTagsFraction.png]]

=====updateTagGrid()=====
This function queries the tag prompts on the page, and uses these data to update the tag heatgrid class and text.

[[File:updateTagGrid.png]]

=====toggleHeatGridRows()=====
This function "Collapses" the heatgrid down to only the two header rows. Note that this function is called both onClick of the heatgrid header, so a user can collapse the grid manually, and automatically when the tag fraction is updated. This ensures that a user visiting the page with all tags complete will see a collapsed heatgrid by default.

[[File:toggleHeatGridRows.png]]

====Stylesheets - grades.scss====
We augmented the stylesheets used in grades so that the new heatgrid could be styled as small as possible to save page real estate. To do so, we added new classes in grades.scss

[[File:stylesheetsGrades_scss.png]]

===Client-Side Implementation: "Alternate View" (view_my_scores.html.erb)===
=====countTotalTags()=====
A simple JS function to count all the tag elements on current page. It will be invoked when the page first being rendered and also when any tags are changed by users.
[[File:countTotalTags.png]]

=====countTaggedTags()=====
A simple JS function to count the number of tagged tags on the current page. It will be invoked when the page first being rendered and when any tags are changed by users.
[[File:countTaggedTags.png]]

=====Additional Logic Check=====
This JS helper dynamically render the page when any tags are changed by users. Since we adopt different logic on two view pages, we added a condition check to tell which page we are on and apply the corresponding logic.
[[File:answer_tag.png]]

==Testing==
Our modifications were mostly on the front end of the code in the app/views and app/assets sections so we decided to create a spec/features test. Our feature test is named view_team_spec.rb and its purpose is to ensure that the table for the heatgrid is generated when accessing the "Your Scores" section. We were able to accomplish this by mostly drawing off of pre-existing feature tests namely peer_review_spec.rb, which had a user access and review another project.

You can run our created feature test with the following command in the expertiza directory. 
''rspec spec/features/view_team_spec.rb''

Expected Output:
1 example, 0 failures

Since we did not make any changes to the model or controller classes we did construct any RSpec tests for any model or controller behavior.

===Navigating to Tag Report===
For manual testing purposes, we logged in as instructor6 and impersonated a student to view the reviews given to that student and be able to tag the reviews and see if the tagged reviews are updated in the heatmap.

 
A good example of functionality for a project with reviews and one with reviews can both be found with student7856. Program 2 has plenty of reviews that can be tagged and untagged and checked that the heatmap functionality is working. Program 1 does not have any reviews to tag and was checked to ensure that nothing broke due to not having any reviews to tag.

===Manual Testing Instructions===
1. Login to Expertiza as "instructor6" with password "password". If hosted locally the url to access Expertiza should be "localhost:3000". 
2. Click on Manage... then "Impersonate User" 
3. Enter "student7856" into the field and click the Impersonate button 
You should see the following page. 
[[File:ImpersonateStudent.PNG]] 
4. Click on "Program 2" 
5. Click on "Your Scores" 
You should see the following page. 
[[File:YourScores.PNG]]
6. You should be able to click on a review score (1-5) and have the table expand, if you tag or remove a tag from a review, you should see the table on the right hand side of the page update to reflect your changes. 
7. Click Back and then click on "Alternate View" 
You should see the following page. 
[[File:AlternateView.PNG]]

===Testing HTML changes using Capybara===
Our feature file test is located at expertiza/spec/features/view_team_spec.rb and tests adding a review and viewing the "Your Scores" page to see the heatmap of review tags. 

The following section of code sets up the tests with factory methods located in spec/factories/factories.rb. It creates a User First and Second who are located on Team 1, a User Third who is located on Team 2. It also maps the third user to review team 1 and the first and second users to review team 2.
''before(:each) do
create(:assignment, name: "TestAssignment", directory_path: 'test_assignment')
create_list(:participant, 3)
create(:assignment_node)
create(:deadline_type, name: "submission")
create(:deadline_type, name: "review")
create(:deadline_type, name: "metareview")
create(:deadline_type, name: "drop_topic")
create(:deadline_type, name: "signup")
create(:deadline_type, name: "team_formation")
create(:deadline_right)
create(:deadline_right, name: 'Late')
create(:deadline_right, name: 'OK')
create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now.in_time_zone + 1.day)
create(:topic)
create(:topic, topic_name: "TestReview")
create(:team_user, user: User.where(role_id: 1).first)
create(:team_user, user: User.where(role_id: 1).second)
create(:assignment_team)
create(:team_user, user: User.where(role_id: 1).third, team: AssignmentTeam.second)
create(:signed_up_team)
create(:signed_up_team, team_id: 2, topic: SignUpTopic.second)
create(:assignment_questionnaire)
create(:question)
create(:submission_record)
create(:submission_record, team_id: 2)
create(:review_response_map, reviewer_id: User.where(role_id: 1).third.id)
create(:review_response_map, reviewer_id: User.where(role_id: 1).first.id, reviewee: AssignmentTeam.second)
create(:review_response_map, reviewer_id: User.where(role_id: 1).second.id, reviewee: AssignmentTeam.second)
create(:review_grade, review_graded_at: Time.now.in_time_zone)
end''

The following section of code executes the common functionality used in the test of logging in as the third user and then navigating to their "Your Scores" page.
''def load_your_scores
login_as(User.where(role_id:1).third.name)
expect(page).to have_content "User: " + User.where(role_id:1).third.name
click_link "Assignments"
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Your scores"
expect(page).to have_content "Alternate View"
click_link "Your scores"
expect(page).to have_content 'Summary Report for assignment: TestAssignment'
end''

The next section of code is the add_review method, which logs in as the first user and adds a review to team 2 which the third user is a part of. This section of code was heavy influenced by "peer_review_spec.rb"
''def add_review
login_as(User.where(role_id:1).first.name)
expect(page).to have_content "User: " + User.where(role_id:1).first.name
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Others' work"
click_link "Others' work"
expect(page).to have_content 'Reviews for "TestAssignment"'
choose "topic_id"
click_button "Request a new submission to review"
click_link "Begin"
fill_in "responses[0][comment]", with: "HelloWorld"
select 3, from: "responses[0][score]"
click_button "Submit Review"
expect(page).to have_content "Your response was successfully saved."
click_link "Logout"
end''

The final section of our test is the actual feature test that checks the heatmap. This section of the code utilizes both of the above defined "load_your_scores" and "add_review" methods to accomplish the task of viewing the heatmap for the third User.
''it "Should contain html target and javascript calls for tag heatgrid" do
# Load Summary Report with no reviews
load_your_scores
expect(page).to have_content "Average peer review score: "
# Add review as first user
click_link "Logout"
add_review
# View Your Scores with one review
load_your_scores
# Check for target to build new heatgrid onto
expect(page.body).to include <nowiki>'<table id="tag_heat_grid" class="tag_heat_grid"></table>'</nowiki>
# Check for Javascript action to generate the heatgrid
expect(page.body).to include 'tagActionOnLoad();'
# Check for Javascript action to turn the heatgrid on and off with answer tag toggle
expect(page.body).to include "$('.tag_heat_grid').toggle();"
end''

==Additional Modifications -- Bugfix in Assignment Model and Grades Controller==
In developing the tagging report, the team noticed a significant, pre-existing view bug on view_team in the beta branch of Expertiza. This bug was causing rows of the review heatgrid to not be populated when the page was rendered. The team is still in communication with the main Expertiza development team to review and verify the validity of these modifications. At the time of this document, all build tests are passing.

'''Screenshot of unpatched bug: '''
[[File:unpatched_view_bug.png|1000px]]

'''Screenshot with patch applied: '''
[[File:patched_view_bug.png|1000px]]

===Origin of Bug===
We discovered a twofold cause of this bug. In part, in the '''grades controller''', calls were being made to an ActiveRecord query, '''@assignment.vary_by_round,''' which was always returning zero. Upon changing this query to use the helper method, '''varying_rubrics_by_round?''' from the '''Assignment''' model, we discovered that a recent change to the logic of '''varying_rubrics_by_round?''' was also causing unexpected behavior. We have included four modifications to our pull request that patch this bug.

===Patches to Assignment model===
In the '''varying_rubrics_by_round?''' method in the '''Assignment''' model, we discovered a recent change to the logic that was causing this method to test whether there were 2 or more '''AssignmentQuestionnaires''' assigned to Round 2 of reviews. Instead, this method should test whether there are 2 or more AssignmentQuestionnaires used on the assignment. We changed this logic to query using <code>used_in_round [1,2]</code> instead of <code>used_in_round 2</code>, which should evaluate '''.size > 1 TRUE''' if each round has a different '''AssignmentQuestionnaire'''.

[[File:assignment_model_patch.png]]

===Patches to Grades Controller===
In the grades controller, queries were being made to <code>@assignment.vary_by_round</code> instead of the helper method, <code>@assignment.varying_rubrics_by_round?</code>. These queries were always returning 0, because that DB field in our DB appears to be 0 for all table rows. We changed the 4 calls from this query to method calls.

[[File:grades_controller_patch.png]]

===Updates to Assignment Tests===
We also discovered an issue with how <code>varying_rubrics_by_round?</code> is tested. It seems, in the unmodified beta branch version, the test that should expect TRUE when rubrics are different for round 1 and 2, is actually testing whether Round 2 has more than 1 rubric assigned to it. We would expect this test to pass when using the old logic in <code>varying_rubrics_by_round?</code>. We modified the TRUE case to assign AssignmentQuestionnaire 1 to round 2, and AssignmentQuestionnaire 2 to round 1.

Also, it seemed that the test for FALSE was testing using one questionnaire in round 1, and a different questionnaire in round 2, which seems logically incorrect for this method. We believe the false case should test for the same rubric across two rounds, in which case the rubrics would not be varying by round. We modified the FALSE case to assign AssignmentQuestionnaire 1 to both round 1 and round 2.

[[File:assignment_spec_patch.png]]

===Updates to Grades Controller Tests===
Lastly, the changes to how <code>varying_rubrics_by_round?</code> is implemented meant that the two tests for the grades controller which rely on this method needed to be updated to expect the new logic. We implemented these updates to these two tests:

[[File:grades_controller_spec.png]]

CSC/ECE 517 Spring 2021 - E2100. Tagging report for students

2021-03-19T20:59:52Z

Jmhurst: /* Navigating to Tag Report */

This page details project documentation for the Spring 2021, "E2100 Tagging report for students" project, which aims to assist students with finding, and completing, incomplete "review tags" on an assignment using a dynamically generated heatgrid.

==Introduction to Expertiza==
[http://http://expertiza.ncsu.edu/ Expertiza] is an [http://en.wikipedia.org/wiki/Open-source_software Open Source] [http://rubyonrails.org/ Rails] application which is used by instructors and students for creating assignments and submitting peer reviews. Expertiza allows the instructor to create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. The Source code of the application can be cloned from [https://github.com/expertiza/expertiza/ Github].

==About Review Tagging==
Review "tags" are a form of feedback on Expertiza where students "tag" (classify) text from peer reviews based on parameters specific to each tag deployment. The parameters can include helpfulness, positivity, suggestions, whether a review offered mitigation, and other parameters depending on the Answer Tag Deployment. These labeled data are then made available to Expertiza researchers for use in developing Natural Language Processing (NLP) / Machine Learning (ML) algorithms. Tagging is only collected for reviews where the text is a sufficient length to be useful as labeled data for NLP research.

==Problem Statement==
It can be difficult for students to find a tag they missed on the Team View page, and other teams are working with ML algorithms to "pre-tag" as many reviews as possible, leading to a granular field of completed/incomplete tags. For example, an assignment with two rounds of reviewing, ten questions per review, twelve reviews, and a 5-parameter tag deployment could contain as few as zero or as many as one thousand, two hundred tag prompts for a single student to complete.

At this time, the only tagging feedback students see are the tag prompts and a Javascript counter with a numeric representation of how many tags have not been completed. In order to find a missed tag, students have to scroll the page and manually search for tags that aren't done.

In order to help students complete all the tags for an assignment, we propose a new, dynamically generated heatgrid on the "Your Scores" view that breaks down reviews and tags by round, question, and review, which uses visual cues to help students find incomplete tags. The heatgrid shows both a total count of "complete" out of "available" tags, and individual tags with color-scaled feedback for completeness.

'''The new heatgrid must handle these requirements, and display visual feedback accordingly:'''
* Single and multiple round reviews
* Reviews with and without tag prompts
* Dynamically update as tags are completed
* Different tag deployments will use different numbers of tags per review
* Use of existing review row-sort functionality
* Not rely on the database for tag data, as AnswerTag database rows may not exist until tags are clicked.
* Give useable feedback for all users (characters as well as color-coding)
* Only be shown when a tags have been deployed to an assignment
* Show the total progress of tagging in the format, "249 out of 315"

==Implementation==
Our proposed solution is a visual feedback aid which exists purely for student users. Tagging data, when input, is stored to the database dynamically using jQuery. For these reasons, and to facilitate dynamic updates, we have chosen to implement this functionality entirely on the client-side of the application using Javascript and jQuery. The presence or absence of tag prompts are detected at page load, and the heat grid is rendered as appropriate. The flowchart below details the new Javascript functions which were implemented to accomplish this goal, and the associated program flow.

'''Your Scores Flow Diagram'''

[[File:View_teamRevA.png]]

'''Heat grid for a two-round review'''

[[File:HeatGridDemo.png|1100px]]

===Code Style===
A note on code style: Because this project utilizes so much Javascript, we tried to keep the code style consistent and true to best practices for both Ruby and Javascript, depending on which language we were working in. Therefore, any code or variables from the Ruby section should use snake_case, and code in the Javascript section should use lowerCamelCase.

===Server-Side Implementation: "Your Scores" and "Alternate View"===
Because most of the logic is implemented on the client-side, the server-side implementation is fairly simple.

===="Your Scores" view (/views/grades/view_team.html.erb)====
We added an empty HTML <<nowiki />table> tag with id and tag information that is used to generate the heatgrid with Javascript.

[[File:view_team_table_tag.png]]

In order to simplify the jQuery which parses the rendered page for tags, we added three numeric and one boolean jQueryable data- fields and a new id field to the td container tag which holds each review text (and the associated tags, if present). We used existing Rails variables to populate these data and id fields. The id is set using "rr" for reviewrow (shortened for brevity) plus the question and review number (j and index). We also created fields for data-round (round number) data-question_num (question number), data-review_num (review number), and data-has_tag (boolean whether tags exist).

A word on style choices: the "data-" is the key queried by jQuery, so a dash is used. The string after the dash is free for us to choose the style, so we used Ruby style to match the convention on the rest of the application.

[[File:tdRailsTag.png]]

Finally, we added a call inside the <code>document.Ready()</code> event handler to tagActionOnLoad(), which starts the rendering code for the client-side Javascript once the page has loaded and tags (if available) are present for counting. We also added a jQuery of our new heat grid table within the onClick for the "Hide Tags" link to ensure our heatgrid is hidden when tag prompts are hidden.

[[File:view_team_javascript.png]]

===="Alternate View" (/views/grades/view_my_scores.html.erb)====

We added function calls to countTotalTags() and countTaggedTags() inside the document.Ready() event handler to calculate the total number of tags and the total number of tagged tags on the page that currently being rendered. We also added div HTML element with id tag_stats and render this element with string "Tag Finished: tagged_tags/total_tags". For example, if there are 370 tags in total on this page and 10 of them has already been tagged, then 10/370 will be displayed. This element will also be updated dynamically when users make tag changes on this page.

[[File:alternate_view_html_code.png]]

===Client-Side Implementation: "Your Scores" (view_team.html.erb)===
The majority of the logic is implemented on the client side. After the page is loaded and the rest of the ReactJS interface is rendered, a dynamically rendered heatgrid is rendered on the "Your Scores" view. A series of functions included in Javascript asset files <code>view_team_in_grades.js</code> and <code>answer_tags.js</code> are called which render the appropriate element for each view. These functions are described below, organized in calling order.

====Initialization and Rendering - view_team_in_grades.js====
=====tagActionOnLoad()=====
Called by <code>document.Ready()</code> event handler, this function first calls <code><nowiki> getTagPrompts()</nowiki></code>, determines if zero exist, and exits if so. If tag prompts exist, it continues to call other functions which query the page for information and render the tag heatgrid.
[[File:tagActionOnLoad.png]]

=====getTagPrompts()=====
A simple Javascript query that collects all the tag prompts as an HTMLCollection for use with other function.

[[File:getTagPrompts.png]]

=====calcTagRatio()=====
A mathematics helper function that uses the tag prompt collection to gather data used in populating the tag heatgrid header ("tag fraction") which reads, for example, "25 out of 350". This function gathers all the numeric data, as well as calculating the completion ratio as a decimal and adapting this decimal to a value that can be used to reference heatgrid color classes (c0 grey, c1 red, c2 orange, c3 yellow, c4 light green, c5 dark green). Note that we avoided using the light green class because it's too easily confused with dark green -- meaning "all done" -- so logic in this function ensures we skip straight from c3 yellow @ .9999 completion, to c5 dark green at 1.0000 completion.

[[File:calcTagRatio.png]]

=====getRowData()=====
This method uses jQuery to retrieve the row containers for all questions in all rounds of this review, whether they contain tag prompts or not. This allows us to populate the heatgrid with 'grey - universal No-symbol' for reviews whose length metric is too low to have a tag prompt associated.

[[File:getRowData.png]]

=====countRounds()=====
This function parses the rows of the review to determine how many rounds of reviews exist in the assignment. This allows us to intelligently print "Round 2 -- Question 3" for multi round reviews, or "Question 3" for single round reviews.

[[File:countRounds.png]]

=====getGridWidth()=====
This function parses the row data to determine how many tag prompts were used in this deployment using a simple selection search for the largest number of tags in a review. Since different tag deployments use different numbers of prompts, the rendering of the heatgrid needs to address this flexibility. Furthermore, the design of this function will allow for future flexibility of tag deployments, i.e., where different reviews in the same assignment or round may have different numbers of tags.

[[File:getGridWidth.png]]

=====drawTagGrid()=====
This is the master function for the rendering of the tag grid. This function sets up tooltip text, calls countRounds() and getGridWidth() and passes control to the three sub-functions, drawHeader(), drawQuestionRow(), and drawReviewRow(), depending on what type of item needs to be rendered next.

[[File:drawTagGrid.png]]

=====drawHeader()=====
This function draws the two-row header of the tag heatgrid which will contain "Tags Completed" and "15 out of 425". It also calls addToolTip() to add the tooltips for expand/contract, and the color legend for the tag fraction row.

[[File:drawHeader.png]]

=====drawQuestionRow()=====
This function is called once per question during the rendering process, and draws the row which will read, "Question 3" or "Round 2 -- Question 3". This also calls addToolTip() to add a color legend tooltip for the body of the heatgrid.

[[File:drawQuestionRow.png]]

=====drawReviewRow()=====
This function draws each row containing the presence, or absence of, tag prompts for each review. These contain R.# (Review.number), and a unicode symbol (set in the global variables). The unicode symbol is to enable accessibility for users who cannot see the red/green spectrum clearly, and will be populated with either a Universal No to mean no tags available, a Warning symbol, to mean tags not done, or a Check-Mark to indicate tags done.

[[File:drawReviewRow.png]]

Lastly, a call is made to updateTagsFraction() as part of the initial render to update the "12 out of 230" header row in the tag grid.

====Updating and Interaction Functions - view_team_in_grades.js====
=====tagActionOnUpdate()=====
Each time a tag prompt is changed, this function is called to update the heatgrid. This function calls several other functions, some of which have been used previously it the rendering step.

[[File:tagActionOnUpdate.png]]

=====updateTagsFraction()=====
This function is used both once in the rendering step, and again each time a tag prompt is changed. This function takes the data from calcTagRatio() and updates the contents and color of the header row in the tag heatgrid ("12 out of 220")

[[File:updateTagsFraction.png]]

=====updateTagGrid()=====
This function queries the tag prompts on the page, and uses these data to update the tag heatgrid class and text.

[[File:updateTagGrid.png]]

=====toggleHeatGridRows()=====
This function "Collapses" the heatgrid down to only the two header rows. Note that this function is called both onClick of the heatgrid header, so a user can collapse the grid manually, and automatically when the tag fraction is updated. This ensures that a user visiting the page with all tags complete will see a collapsed heatgrid by default.

[[File:toggleHeatGridRows.png]]

====Stylesheets - grades.scss====
We augmented the stylesheets used in grades so that the new heatgrid could be styled as small as possible to save page real estate. To do so, we added new classes in grades.scss

[[File:stylesheetsGrades_scss.png]]

===Client-Side Implementation: "Alternate View" (view_my_scores.html.erb)===
=====countTotalTags()=====
A simple JS function to count all the tag elements on current page. It will be invoked when the page first being rendered and also when any tags are changed by users.
[[File:countTotalTags.png]]

=====countTaggedTags()=====
A simple JS function to count the number of tagged tags on the current page. It will be invoked when the page first being rendered and when any tags are changed by users.
[[File:countTaggedTags.png]]

=====Additional Logic Check=====
This JS helper dynamically render the page when any tags are changed by users. Since we adopt different logic on two view pages, we added a condition check to tell which page we are on and apply the corresponding logic.
[[File:answer_tag.png]]

==Testing==
Our modifications were mostly on the front end of the code in the app/views and app/assets sections so we decided to create a spec/features test. Our feature test is named view_team_spec.rb and its purpose is to ensure that the table for the heatgrid is generated when accessing the "Your Scores" section. We were able to accomplish this by mostly drawing off of pre-existing feature tests namely peer_review_spec.rb, which had a user access and review another project.

You can run our created feature test with the following command in the expertiza directory.
rspec spec/features/view_team_spec.rb

Expected Output:
1 example, 0 failures

Since we did not make any changes to the model or controller classes we did construct any RSpec tests for any model or controller behavior.

===Navigating to Tag Report===
For manual testing purposes, we logged in as instructor6 and impersonated a student to view the reviews given to that student and be able to tag the reviews and see if the tagged reviews are updated in the heatmap.

 
A good example of functionality for a project with reviews and one with reviews can both be found with student7856. Program 2 has plenty of reviews that can be tagged and untagged and checked that the heatmap functionality is working. Program 1 does not have any reviews to tag and was checked to ensure that nothing broke due to not having any reviews to tag.

===Manual Testing Instructions===
1. Login to Expertiza as "instructor6" with password "password". If hosted locally the url to access Expertiza should be "localhost:3000". 
2. Click on Manage... then "Impersonate User" 
3. Enter "student7856" into the field and click the Impersonate button 
You should see the following page. 
[[File:ImpersonateStudent.PNG]] 
4. Click on "Program 2" 
5. Click on "Your Scores" 
You should see the following page. 
[[File:YourScores.PNG]]
6. You should be able to click on a review score (1-5) and have the table expand, if you tag or remove a tag from a review, you should see the table on the right hand side of the page update to reflect your changes. 
7. Click Back and then click on "Alternate View" 
You should see the following page. 
[[File:AlternateView.PNG]]

===Testing HTML changes using Capybara===
Our feature file test is located at expertiza/spec/features/view_team_spec.rb and tests adding a review and viewing the "Your Scores" page to see the heatmap of review tags. 

The following section of code sets up the tests with factory methods located in spec/factories/factories.rb. It creates a User First and Second who are located on Team 1, a User Third who is located on Team 2. It also maps the third user to review team 1 and the first and second users to review team 2.
''before(:each) do
create(:assignment, name: "TestAssignment", directory_path: 'test_assignment')
create_list(:participant, 3)
create(:assignment_node)
create(:deadline_type, name: "submission")
create(:deadline_type, name: "review")
create(:deadline_type, name: "metareview")
create(:deadline_type, name: "drop_topic")
create(:deadline_type, name: "signup")
create(:deadline_type, name: "team_formation")
create(:deadline_right)
create(:deadline_right, name: 'Late')
create(:deadline_right, name: 'OK')
create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now.in_time_zone + 1.day)
create(:topic)
create(:topic, topic_name: "TestReview")
create(:team_user, user: User.where(role_id: 1).first)
create(:team_user, user: User.where(role_id: 1).second)
create(:assignment_team)
create(:team_user, user: User.where(role_id: 1).third, team: AssignmentTeam.second)
create(:signed_up_team)
create(:signed_up_team, team_id: 2, topic: SignUpTopic.second)
create(:assignment_questionnaire)
create(:question)
create(:submission_record)
create(:submission_record, team_id: 2)
create(:review_response_map, reviewer_id: User.where(role_id: 1).third.id)
create(:review_response_map, reviewer_id: User.where(role_id: 1).first.id, reviewee: AssignmentTeam.second)
create(:review_response_map, reviewer_id: User.where(role_id: 1).second.id, reviewee: AssignmentTeam.second)
create(:review_grade, review_graded_at: Time.now.in_time_zone)
end''

The following section of code executes the common functionality used in the test of logging in as the third user and then navigating to their "Your Scores" page.
''def load_your_scores
login_as(User.where(role_id:1).third.name)
expect(page).to have_content "User: " + User.where(role_id:1).third.name
click_link "Assignments"
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Your scores"
expect(page).to have_content "Alternate View"
click_link "Your scores"
expect(page).to have_content 'Summary Report for assignment: TestAssignment'
end''

The next section of code is the add_review method, which logs in as the first user and adds a review to team 2 which the third user is a part of. This section of code was heavy influenced by "peer_review_spec.rb"
''def add_review
login_as(User.where(role_id:1).first.name)
expect(page).to have_content "User: " + User.where(role_id:1).first.name
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Others' work"
click_link "Others' work"
expect(page).to have_content 'Reviews for "TestAssignment"'
choose "topic_id"
click_button "Request a new submission to review"
click_link "Begin"
fill_in "responses[0][comment]", with: "HelloWorld"
select 3, from: "responses[0][score]"
click_button "Submit Review"
expect(page).to have_content "Your response was successfully saved."
click_link "Logout"
end''

The final section of our test is the actual feature test that checks the heatmap. This section of the code utilizes both of the above defined "load_your_scores" and "add_review" methods to accomplish the task of viewing the heatmap for the third User.
''it "Should contain html target and javascript calls for tag heatgrid" do
# Load Summary Report with no reviews
load_your_scores
expect(page).to have_content "Average peer review score: "
# Add review as first user
click_link "Logout"
add_review
# View Your Scores with one review
load_your_scores
# Check for target to build new heatgrid onto
expect(page.body).to include <nowiki>'<table id="tag_heat_grid" class="tag_heat_grid"></table>'</nowiki>
# Check for Javascript action to generate the heatgrid
expect(page.body).to include 'tagActionOnLoad();'
# Check for Javascript action to turn the heatgrid on and off with answer tag toggle
expect(page.body).to include "$('.tag_heat_grid').toggle();"
end''

==Additional Modifications -- Bugfix in Assignment Model and Grades Controller==
In developing the tagging report, the team noticed a significant, pre-existing view bug on view_team in the beta branch of Expertiza. This bug was causing rows of the review heatgrid to not be populated when the page was rendered. The team is still in communication with the main Expertiza development team to review and verify the validity of these modifications. At the time of this document, all build tests are passing.

'''Screenshot of unpatched bug: '''
[[File:unpatched_view_bug.png|1000px]]

'''Screenshot with patch applied: '''
[[File:patched_view_bug.png|1000px]]

===Origin of Bug===
We discovered a twofold cause of this bug. In part, in the '''grades controller''', calls were being made to an ActiveRecord query, '''@assignment.vary_by_round,''' which was always returning zero. Upon changing this query to use the helper method, '''varying_rubrics_by_round?''' from the '''Assignment''' model, we discovered that a recent change to the logic of '''varying_rubrics_by_round?''' was also causing unexpected behavior. We have included four modifications to our pull request that patch this bug.

===Patches to Assignment model===
In the '''varying_rubrics_by_round?''' method in the '''Assignment''' model, we discovered a recent change to the logic that was causing this method to test whether there were 2 or more '''AssignmentQuestionnaires''' assigned to Round 2 of reviews. Instead, this method should test whether there are 2 or more AssignmentQuestionnaires used on the assignment. We changed this logic to query using <code>used_in_round [1,2]</code> instead of <code>used_in_round 2</code>, which should evaluate '''.size > 1 TRUE''' if each round has a different '''AssignmentQuestionnaire'''.

[[File:assignment_model_patch.png]]

===Patches to Grades Controller===
In the grades controller, queries were being made to <code>@assignment.vary_by_round</code> instead of the helper method, <code>@assignment.varying_rubrics_by_round?</code>. These queries were always returning 0, because that DB field in our DB appears to be 0 for all table rows. We changed the 4 calls from this query to method calls.

[[File:grades_controller_patch.png]]

===Updates to Assignment Tests===
We also discovered an issue with how <code>varying_rubrics_by_round?</code> is tested. It seems, in the unmodified beta branch version, the test that should expect TRUE when rubrics are different for round 1 and 2, is actually testing whether Round 2 has more than 1 rubric assigned to it. We would expect this test to pass when using the old logic in <code>varying_rubrics_by_round?</code>. We modified the TRUE case to assign AssignmentQuestionnaire 1 to round 2, and AssignmentQuestionnaire 2 to round 1.

Also, it seemed that the test for FALSE was testing using one questionnaire in round 1, and a different questionnaire in round 2, which seems logically incorrect for this method. We believe the false case should test for the same rubric across two rounds, in which case the rubrics would not be varying by round. We modified the FALSE case to assign AssignmentQuestionnaire 1 to both round 1 and round 2.

[[File:assignment_spec_patch.png]]

===Updates to Grades Controller Tests===
Lastly, the changes to how <code>varying_rubrics_by_round?</code> is implemented meant that the two tests for the grades controller which rely on this method needed to be updated to expect the new logic. We implemented these updates to these two tests:

[[File:grades_controller_spec.png]]

CSC/ECE 517 Spring 2021 - E2100. Tagging report for students

2021-03-19T20:56:00Z

Jmhurst: /* Testing HTML changes using Capybara */

This page details project documentation for the Spring 2021, "E2100 Tagging report for students" project, which aims to assist students with finding, and completing, incomplete "review tags" on an assignment using a dynamically generated heatgrid.

==Introduction to Expertiza==
[http://http://expertiza.ncsu.edu/ Expertiza] is an [http://en.wikipedia.org/wiki/Open-source_software Open Source] [http://rubyonrails.org/ Rails] application which is used by instructors and students for creating assignments and submitting peer reviews. Expertiza allows the instructor to create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. The Source code of the application can be cloned from [https://github.com/expertiza/expertiza/ Github].

==About Review Tagging==
Review "tags" are a form of feedback on Expertiza where students "tag" (classify) text from peer reviews based on parameters specific to each tag deployment. The parameters can include helpfulness, positivity, suggestions, whether a review offered mitigation, and other parameters depending on the Answer Tag Deployment. These labeled data are then made available to Expertiza researchers for use in developing Natural Language Processing (NLP) / Machine Learning (ML) algorithms. Tagging is only collected for reviews where the text is a sufficient length to be useful as labeled data for NLP research.

==Problem Statement==
It can be difficult for students to find a tag they missed on the Team View page, and other teams are working with ML algorithms to "pre-tag" as many reviews as possible, leading to a granular field of completed/incomplete tags. For example, an assignment with two rounds of reviewing, ten questions per review, twelve reviews, and a 5-parameter tag deployment could contain as few as zero or as many as one thousand, two hundred tag prompts for a single student to complete.

At this time, the only tagging feedback students see are the tag prompts and a Javascript counter with a numeric representation of how many tags have not been completed. In order to find a missed tag, students have to scroll the page and manually search for tags that aren't done.

In order to help students complete all the tags for an assignment, we propose a new, dynamically generated heatgrid on the "Your Scores" view that breaks down reviews and tags by round, question, and review, which uses visual cues to help students find incomplete tags. The heatgrid shows both a total count of "complete" out of "available" tags, and individual tags with color-scaled feedback for completeness.

'''The new heatgrid must handle these requirements, and display visual feedback accordingly:'''
* Single and multiple round reviews
* Reviews with and without tag prompts
* Dynamically update as tags are completed
* Different tag deployments will use different numbers of tags per review
* Use of existing review row-sort functionality
* Not rely on the database for tag data, as AnswerTag database rows may not exist until tags are clicked.
* Give useable feedback for all users (characters as well as color-coding)
* Only be shown when a tags have been deployed to an assignment
* Show the total progress of tagging in the format, "249 out of 315"

==Implementation==
Our proposed solution is a visual feedback aid which exists purely for student users. Tagging data, when input, is stored to the database dynamically using jQuery. For these reasons, and to facilitate dynamic updates, we have chosen to implement this functionality entirely on the client-side of the application using Javascript and jQuery. The presence or absence of tag prompts are detected at page load, and the heat grid is rendered as appropriate. The flowchart below details the new Javascript functions which were implemented to accomplish this goal, and the associated program flow.

'''Your Scores Flow Diagram'''

[[File:View_teamRevA.png]]

'''Heat grid for a two-round review'''

[[File:HeatGridDemo.png|1100px]]

===Code Style===
A note on code style: Because this project utilizes so much Javascript, we tried to keep the code style consistent and true to best practices for both Ruby and Javascript, depending on which language we were working in. Therefore, any code or variables from the Ruby section should use snake_case, and code in the Javascript section should use lowerCamelCase.

===Server-Side Implementation: "Your Scores" and "Alternate View"===
Because most of the logic is implemented on the client-side, the server-side implementation is fairly simple.

===="Your Scores" view (/views/grades/view_team.html.erb)====
We added an empty HTML <<nowiki />table> tag with id and tag information that is used to generate the heatgrid with Javascript.

[[File:view_team_table_tag.png]]

In order to simplify the jQuery which parses the rendered page for tags, we added three numeric and one boolean jQueryable data- fields and a new id field to the td container tag which holds each review text (and the associated tags, if present). We used existing Rails variables to populate these data and id fields. The id is set using "rr" for reviewrow (shortened for brevity) plus the question and review number (j and index). We also created fields for data-round (round number) data-question_num (question number), data-review_num (review number), and data-has_tag (boolean whether tags exist).

A word on style choices: the "data-" is the key queried by jQuery, so a dash is used. The string after the dash is free for us to choose the style, so we used Ruby style to match the convention on the rest of the application.

[[File:tdRailsTag.png]]

Finally, we added a call inside the <code>document.Ready()</code> event handler to tagActionOnLoad(), which starts the rendering code for the client-side Javascript once the page has loaded and tags (if available) are present for counting. We also added a jQuery of our new heat grid table within the onClick for the "Hide Tags" link to ensure our heatgrid is hidden when tag prompts are hidden.

[[File:view_team_javascript.png]]

===="Alternate View" (/views/grades/view_my_scores.html.erb)====

We added function calls to countTotalTags() and countTaggedTags() inside the document.Ready() event handler to calculate the total number of tags and the total number of tagged tags on the page that currently being rendered. We also added div HTML element with id tag_stats and render this element with string "Tag Finished: tagged_tags/total_tags". For example, if there are 370 tags in total on this page and 10 of them has already been tagged, then 10/370 will be displayed. This element will also be updated dynamically when users make tag changes on this page.

[[File:alternate_view_html_code.png]]

===Client-Side Implementation: "Your Scores" (view_team.html.erb)===
The majority of the logic is implemented on the client side. After the page is loaded and the rest of the ReactJS interface is rendered, a dynamically rendered heatgrid is rendered on the "Your Scores" view. A series of functions included in Javascript asset files <code>view_team_in_grades.js</code> and <code>answer_tags.js</code> are called which render the appropriate element for each view. These functions are described below, organized in calling order.

====Initialization and Rendering - view_team_in_grades.js====
=====tagActionOnLoad()=====
Called by <code>document.Ready()</code> event handler, this function first calls <code><nowiki> getTagPrompts()</nowiki></code>, determines if zero exist, and exits if so. If tag prompts exist, it continues to call other functions which query the page for information and render the tag heatgrid.
[[File:tagActionOnLoad.png]]

=====getTagPrompts()=====
A simple Javascript query that collects all the tag prompts as an HTMLCollection for use with other function.

[[File:getTagPrompts.png]]

=====calcTagRatio()=====
A mathematics helper function that uses the tag prompt collection to gather data used in populating the tag heatgrid header ("tag fraction") which reads, for example, "25 out of 350". This function gathers all the numeric data, as well as calculating the completion ratio as a decimal and adapting this decimal to a value that can be used to reference heatgrid color classes (c0 grey, c1 red, c2 orange, c3 yellow, c4 light green, c5 dark green). Note that we avoided using the light green class because it's too easily confused with dark green -- meaning "all done" -- so logic in this function ensures we skip straight from c3 yellow @ .9999 completion, to c5 dark green at 1.0000 completion.

[[File:calcTagRatio.png]]

=====getRowData()=====
This method uses jQuery to retrieve the row containers for all questions in all rounds of this review, whether they contain tag prompts or not. This allows us to populate the heatgrid with 'grey - universal No-symbol' for reviews whose length metric is too low to have a tag prompt associated.

[[File:getRowData.png]]

=====countRounds()=====
This function parses the rows of the review to determine how many rounds of reviews exist in the assignment. This allows us to intelligently print "Round 2 -- Question 3" for multi round reviews, or "Question 3" for single round reviews.

[[File:countRounds.png]]

=====getGridWidth()=====
This function parses the row data to determine how many tag prompts were used in this deployment using a simple selection search for the largest number of tags in a review. Since different tag deployments use different numbers of prompts, the rendering of the heatgrid needs to address this flexibility. Furthermore, the design of this function will allow for future flexibility of tag deployments, i.e., where different reviews in the same assignment or round may have different numbers of tags.

[[File:getGridWidth.png]]

=====drawTagGrid()=====
This is the master function for the rendering of the tag grid. This function sets up tooltip text, calls countRounds() and getGridWidth() and passes control to the three sub-functions, drawHeader(), drawQuestionRow(), and drawReviewRow(), depending on what type of item needs to be rendered next.

[[File:drawTagGrid.png]]

=====drawHeader()=====
This function draws the two-row header of the tag heatgrid which will contain "Tags Completed" and "15 out of 425". It also calls addToolTip() to add the tooltips for expand/contract, and the color legend for the tag fraction row.

[[File:drawHeader.png]]

=====drawQuestionRow()=====
This function is called once per question during the rendering process, and draws the row which will read, "Question 3" or "Round 2 -- Question 3". This also calls addToolTip() to add a color legend tooltip for the body of the heatgrid.

[[File:drawQuestionRow.png]]

=====drawReviewRow()=====
This function draws each row containing the presence, or absence of, tag prompts for each review. These contain R.# (Review.number), and a unicode symbol (set in the global variables). The unicode symbol is to enable accessibility for users who cannot see the red/green spectrum clearly, and will be populated with either a Universal No to mean no tags available, a Warning symbol, to mean tags not done, or a Check-Mark to indicate tags done.

[[File:drawReviewRow.png]]

Lastly, a call is made to updateTagsFraction() as part of the initial render to update the "12 out of 230" header row in the tag grid.

====Updating and Interaction Functions - view_team_in_grades.js====
=====tagActionOnUpdate()=====
Each time a tag prompt is changed, this function is called to update the heatgrid. This function calls several other functions, some of which have been used previously it the rendering step.

[[File:tagActionOnUpdate.png]]

=====updateTagsFraction()=====
This function is used both once in the rendering step, and again each time a tag prompt is changed. This function takes the data from calcTagRatio() and updates the contents and color of the header row in the tag heatgrid ("12 out of 220")

[[File:updateTagsFraction.png]]

=====updateTagGrid()=====
This function queries the tag prompts on the page, and uses these data to update the tag heatgrid class and text.

[[File:updateTagGrid.png]]

=====toggleHeatGridRows()=====
This function "Collapses" the heatgrid down to only the two header rows. Note that this function is called both onClick of the heatgrid header, so a user can collapse the grid manually, and automatically when the tag fraction is updated. This ensures that a user visiting the page with all tags complete will see a collapsed heatgrid by default.

[[File:toggleHeatGridRows.png]]

====Stylesheets - grades.scss====
We augmented the stylesheets used in grades so that the new heatgrid could be styled as small as possible to save page real estate. To do so, we added new classes in grades.scss

[[File:stylesheetsGrades_scss.png]]

===Client-Side Implementation: "Alternate View" (view_my_scores.html.erb)===
=====countTotalTags()=====
A simple JS function to count all the tag elements on current page. It will be invoked when the page first being rendered and also when any tags are changed by users.
[[File:countTotalTags.png]]

=====countTaggedTags()=====
A simple JS function to count the number of tagged tags on the current page. It will be invoked when the page first being rendered and when any tags are changed by users.
[[File:countTaggedTags.png]]

=====Additional Logic Check=====
This JS helper dynamically render the page when any tags are changed by users. Since we adopt different logic on two view pages, we added a condition check to tell which page we are on and apply the corresponding logic.
[[File:answer_tag.png]]

==Testing==
Our modifications were mostly on the front end of the code in the app/views and app/assets sections so we decided to create a spec/features test. Our feature test is named view_team_spec.rb and its purpose is to ensure that the table for the heatgrid is generated when accessing the "Your Scores" section. We were able to accomplish this by mostly drawing off of pre-existing feature tests namely peer_review_spec.rb, which had a user access and review another project.

You can run our created feature test with the following command in the expertiza directory.
rspec spec/features/view_team_spec.rb

Expected Output:
1 example, 0 failures

Since we did not make any changes to the model or controller classes we did construct any RSpec tests for any model or controller behavior.

===Navigating to Tag Report===
For manual testing purposes, we logged in as instructor6 and impersonated a student to view the reviews given to that student and be able to tag the reviews and see if the tagged reviews are updated in the heatmap.

===Manual Testing Instructions===
1. Login to Expertiza as "instructor6" with password "password". If hosted locally the url to access Expertiza should be "localhost:3000". 
2. Click on Manage... then "Impersonate User" 
3. Enter "student7856" into the field and click the Impersonate button 
You should see the following page. 
[[File:ImpersonateStudent.PNG]] 
4. Click on "Program 2" 
5. Click on "Your Scores" 
You should see the following page. 
[[File:YourScores.PNG]]
6. You should be able to click on a review score (1-5) and have the table expand, if you tag or remove a tag from a review, you should see the table on the right hand side of the page update to reflect your changes. 
7. Click Back and then click on "Alternate View" 
You should see the following page. 
[[File:AlternateView.PNG]]

===Testing HTML changes using Capybara===
Our feature file test is located at expertiza/spec/features/view_team_spec.rb and tests adding a review and viewing the "Your Scores" page to see the heatmap of review tags. 

The following section of code sets up the tests with factory methods located in spec/factories/factories.rb. It creates a User First and Second who are located on Team 1, a User Third who is located on Team 2. It also maps the third user to review team 1 and the first and second users to review team 2.
''before(:each) do
create(:assignment, name: "TestAssignment", directory_path: 'test_assignment')
create_list(:participant, 3)
create(:assignment_node)
create(:deadline_type, name: "submission")
create(:deadline_type, name: "review")
create(:deadline_type, name: "metareview")
create(:deadline_type, name: "drop_topic")
create(:deadline_type, name: "signup")
create(:deadline_type, name: "team_formation")
create(:deadline_right)
create(:deadline_right, name: 'Late')
create(:deadline_right, name: 'OK')
create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now.in_time_zone + 1.day)
create(:topic)
create(:topic, topic_name: "TestReview")
create(:team_user, user: User.where(role_id: 1).first)
create(:team_user, user: User.where(role_id: 1).second)
create(:assignment_team)
create(:team_user, user: User.where(role_id: 1).third, team: AssignmentTeam.second)
create(:signed_up_team)
create(:signed_up_team, team_id: 2, topic: SignUpTopic.second)
create(:assignment_questionnaire)
create(:question)
create(:submission_record)
create(:submission_record, team_id: 2)
create(:review_response_map, reviewer_id: User.where(role_id: 1).third.id)
create(:review_response_map, reviewer_id: User.where(role_id: 1).first.id, reviewee: AssignmentTeam.second)
create(:review_response_map, reviewer_id: User.where(role_id: 1).second.id, reviewee: AssignmentTeam.second)
create(:review_grade, review_graded_at: Time.now.in_time_zone)
end''

The following section of code executes the common functionality used in the test of logging in as the third user and then navigating to their "Your Scores" page.
''def load_your_scores
login_as(User.where(role_id:1).third.name)
expect(page).to have_content "User: " + User.where(role_id:1).third.name
click_link "Assignments"
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Your scores"
expect(page).to have_content "Alternate View"
click_link "Your scores"
expect(page).to have_content 'Summary Report for assignment: TestAssignment'
end''

The next section of code is the add_review method, which logs in as the first user and adds a review to team 2 which the third user is a part of. This section of code was heavy influenced by "peer_review_spec.rb"
''def add_review
login_as(User.where(role_id:1).first.name)
expect(page).to have_content "User: " + User.where(role_id:1).first.name
expect(page).to have_content "TestAssignment"
click_link "TestAssignment"
expect(page).to have_content "Submit or Review work for TestAssignment"
expect(page).to have_content "Others' work"
click_link "Others' work"
expect(page).to have_content 'Reviews for "TestAssignment"'
choose "topic_id"
click_button "Request a new submission to review"
click_link "Begin"
fill_in "responses[0][comment]", with: "HelloWorld"
select 3, from: "responses[0][score]"
click_button "Submit Review"
expect(page).to have_content "Your response was successfully saved."
click_link "Logout"
end''

The final section of our test is the actual feature test that checks the heatmap. This section of the code utilizes both of the above defined "load_your_scores" and "add_review" methods to accomplish the task of viewing the heatmap for the third User.
''it "Should contain html target and javascript calls for tag heatgrid" do
# Load Summary Report with no reviews
load_your_scores
expect(page).to have_content "Average peer review score: "
# Add review as first user
click_link "Logout"
add_review
# View Your Scores with one review
load_your_scores
# Check for target to build new heatgrid onto
expect(page.body).to include <nowiki>'<table id="tag_heat_grid" class="tag_heat_grid"></table>'</nowiki>
# Check for Javascript action to generate the heatgrid
expect(page.body).to include 'tagActionOnLoad();'
# Check for Javascript action to turn the heatgrid on and off with answer tag toggle
expect(page.body).to include "$('.tag_heat_grid').toggle();"
end''

==Additional Modifications -- Bugfix in Assignment Model and Grades Controller==
In developing the tagging report, the team noticed a significant, pre-existing view bug on view_team in the beta branch of Expertiza. This bug was causing rows of the review heatgrid to not be populated when the page was rendered. The team is still in communication with the main Expertiza development team to review and verify the validity of these modifications. At the time of this document, all build tests are passing.

'''Screenshot of unpatched bug: '''
[[File:unpatched_view_bug.png|1000px]]

'''Screenshot with patch applied: '''
[[File:patched_view_bug.png|1000px]]

===Origin of Bug===
We discovered a twofold cause of this bug. In part, in the '''grades controller''', calls were being made to an ActiveRecord query, '''@assignment.vary_by_round,''' which was always returning zero. Upon changing this query to use the helper method, '''varying_rubrics_by_round?''' from the '''Assignment''' model, we discovered that a recent change to the logic of '''varying_rubrics_by_round?''' was also causing unexpected behavior. We have included four modifications to our pull request that patch this bug.

===Patches to Assignment model===
In the '''varying_rubrics_by_round?''' method in the '''Assignment''' model, we discovered a recent change to the logic that was causing this method to test whether there were 2 or more '''AssignmentQuestionnaires''' assigned to Round 2 of reviews. Instead, this method should test whether there are 2 or more AssignmentQuestionnaires used on the assignment. We changed this logic to query using <code>used_in_round [1,2]</code> instead of <code>used_in_round 2</code>, which should evaluate '''.size > 1 TRUE''' if each round has a different '''AssignmentQuestionnaire'''.

[[File:assignment_model_patch.png]]

===Patches to Grades Controller===
In the grades controller, queries were being made to <code>@assignment.vary_by_round</code> instead of the helper method, <code>@assignment.varying_rubrics_by_round?</code>. These queries were always returning 0, because that DB field in our DB appears to be 0 for all table rows. We changed the 4 calls from this query to method calls.

[[File:grades_controller_patch.png]]

===Updates to Assignment Tests===
We also discovered an issue with how <code>varying_rubrics_by_round?</code> is tested. It seems, in the unmodified beta branch version, the test that should expect TRUE when rubrics are different for round 1 and 2, is actually testing whether Round 2 has more than 1 rubric assigned to it. We would expect this test to pass when using the old logic in <code>varying_rubrics_by_round?</code>. We modified the TRUE case to assign AssignmentQuestionnaire 1 to round 2, and AssignmentQuestionnaire 2 to round 1.

Also, it seemed that the test for FALSE was testing using one questionnaire in round 1, and a different questionnaire in round 2, which seems logically incorrect for this method. We believe the false case should test for the same rubric across two rounds, in which case the rubrics would not be varying by round. We modified the FALSE case to assign AssignmentQuestionnaire 1 to both round 1 and round 2.

[[File:assignment_spec_patch.png]]

===Updates to Grades Controller Tests===
Lastly, the changes to how <code>varying_rubrics_by_round?</code> is implemented meant that the two tests for the grades controller which rely on this method needed to be updated to expect the new logic. We implemented these updates to these two tests:

[[File:grades_controller_spec.png]]

CSC/ECE 517 Spring 2021 - E2100. Tagging report for students

2021-03-19T20:43:45Z

Jmhurst: /* Testing */ First changes

This page details project documentation for the Spring 2021, "E2100 Tagging report for students" project, which aims to assist students with finding, and completing, incomplete "review tags" on an assignment using a dynamically generated heatgrid.

==Introduction to Expertiza==
[http://http://expertiza.ncsu.edu/ Expertiza] is an [http://en.wikipedia.org/wiki/Open-source_software Open Source] [http://rubyonrails.org/ Rails] application which is used by instructors and students for creating assignments and submitting peer reviews. Expertiza allows the instructor to create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. The Source code of the application can be cloned from [https://github.com/expertiza/expertiza/ Github].

==About Review Tagging==
Review "tags" are a form of feedback on Expertiza where students "tag" (classify) text from peer reviews based on parameters specific to each tag deployment. The parameters can include helpfulness, positivity, suggestions, whether a review offered mitigation, and other parameters depending on the Answer Tag Deployment. These labeled data are then made available to Expertiza researchers for use in developing Natural Language Processing (NLP) / Machine Learning (ML) algorithms. Tagging is only collected for reviews where the text is a sufficient length to be useful as labeled data for NLP research.

==Problem Statement==
It can be difficult for students to find a tag they missed on the Team View page, and other teams are working with ML algorithms to "pre-tag" as many reviews as possible, leading to a granular field of completed/incomplete tags. For example, an assignment with two rounds of reviewing, ten questions per review, twelve reviews, and a 5-parameter tag deployment could contain as few as zero or as many as one thousand, two hundred tag prompts for a single student to complete.

At this time, the only tagging feedback students see are the tag prompts and a Javascript counter with a numeric representation of how many tags have not been completed. In order to find a missed tag, students have to scroll the page and manually search for tags that aren't done.

In order to help students complete all the tags for an assignment, we propose a new, dynamically generated heatgrid on the "Your Scores" view that breaks down reviews and tags by round, question, and review, which uses visual cues to help students find incomplete tags. The heatgrid shows both a total count of "complete" out of "available" tags, and individual tags with color-scaled feedback for completeness.

'''The new heatgrid must handle these requirements, and display visual feedback accordingly:'''
* Single and multiple round reviews
* Reviews with and without tag prompts
* Dynamically update as tags are completed
* Different tag deployments will use different numbers of tags per review
* Use of existing review row-sort functionality
* Not rely on the database for tag data, as AnswerTag database rows may not exist until tags are clicked.
* Give useable feedback for all users (characters as well as color-coding)
* Only be shown when a tags have been deployed to an assignment
* Show the total progress of tagging in the format, "249 out of 315"

==Implementation==
Our proposed solution is a visual feedback aid which exists purely for student users. Tagging data, when input, is stored to the database dynamically using jQuery. For these reasons, and to facilitate dynamic updates, we have chosen to implement this functionality entirely on the client-side of the application using Javascript and jQuery. The presence or absence of tag prompts are detected at page load, and the heat grid is rendered as appropriate. The flowchart below details the new Javascript functions which were implemented to accomplish this goal, and the associated program flow.

'''Your Scores Flow Diagram'''

[[File:View_teamRevA.png]]

'''Heat grid for a two-round review'''

[[File:HeatGridDemo.png|1100px]]

===Code Style===
A note on code style: Because this project utilizes so much Javascript, we tried to keep the code style consistent and true to best practices for both Ruby and Javascript, depending on which language we were working in. Therefore, any code or variables from the Ruby section should use snake_case, and code in the Javascript section should use lowerCamelCase.

===Server-Side Implementation: "Your Scores" and "Alternate View"===
Because most of the logic is implemented on the client-side, the server-side implementation is fairly simple.

===="Your Scores" view (/views/grades/view_team.html.erb)====
We added an empty HTML <<nowiki />table> tag with id and tag information that is used to generate the heatgrid with Javascript.

[[File:view_team_table_tag.png]]

In order to simplify the jQuery which parses the rendered page for tags, we added three numeric and one boolean jQueryable data- fields and a new id field to the td container tag which holds each review text (and the associated tags, if present). We used existing Rails variables to populate these data and id fields. The id is set using "rr" for reviewrow (shortened for brevity) plus the question and review number (j and index). We also created fields for data-round (round number) data-question_num (question number), data-review_num (review number), and data-has_tag (boolean whether tags exist).

A word on style choices: the "data-" is the key queried by jQuery, so a dash is used. The string after the dash is free for us to choose the style, so we used Ruby style to match the convention on the rest of the application.

[[File:tdRailsTag.png]]

Finally, we added a call inside the <code>document.Ready()</code> event handler to tagActionOnLoad(), which starts the rendering code for the client-side Javascript once the page has loaded and tags (if available) are present for counting. We also added a jQuery of our new heat grid table within the onClick for the "Hide Tags" link to ensure our heatgrid is hidden when tag prompts are hidden.

[[File:view_team_javascript.png]]

===="Alternate View" (/views/grades/view_my_scores.html.erb)====

We added function calls to countTotalTags() and countTaggedTags() inside the document.Ready() event handler to calculate the total number of tags and the total number of tagged tags on the page that currently being rendered. We also added div HTML element with id tag_stats and render this element with string "Tag Finished: tagged_tags/total_tags". For example, if there are 370 tags in total on this page and 10 of them has already been tagged, then 10/370 will be displayed. This element will also be updated dynamically when users make tag changes on this page.

[[File:alternate_view_html_code.png]]

===Client-Side Implementation: "Your Scores" (view_team.html.erb)===
The majority of the logic is implemented on the client side. After the page is loaded and the rest of the ReactJS interface is rendered, a dynamically rendered heatgrid is rendered on the "Your Scores" view. A series of functions included in Javascript asset files <code>view_team_in_grades.js</code> and <code>answer_tags.js</code> are called which render the appropriate element for each view. These functions are described below, organized in calling order.

====Initialization and Rendering - view_team_in_grades.js====
=====tagActionOnLoad()=====
Called by <code>document.Ready()</code> event handler, this function first calls <code><nowiki> getTagPrompts()</nowiki></code>, determines if zero exist, and exits if so. If tag prompts exist, it continues to call other functions which query the page for information and render the tag heatgrid.
[[File:tagActionOnLoad.png]]

=====getTagPrompts()=====
A simple Javascript query that collects all the tag prompts as an HTMLCollection for use with other function.

[[File:getTagPrompts.png]]

=====calcTagRatio()=====
A mathematics helper function that uses the tag prompt collection to gather data used in populating the tag heatgrid header ("tag fraction") which reads, for example, "25 out of 350". This function gathers all the numeric data, as well as calculating the completion ratio as a decimal and adapting this decimal to a value that can be used to reference heatgrid color classes (c0 grey, c1 red, c2 orange, c3 yellow, c4 light green, c5 dark green). Note that we avoided using the light green class because it's too easily confused with dark green -- meaning "all done" -- so logic in this function ensures we skip straight from c3 yellow @ .9999 completion, to c5 dark green at 1.0000 completion.

[[File:calcTagRatio.png]]

=====getRowData()=====
This method uses jQuery to retrieve the row containers for all questions in all rounds of this review, whether they contain tag prompts or not. This allows us to populate the heatgrid with 'grey - universal No-symbol' for reviews whose length metric is too low to have a tag prompt associated.

[[File:getRowData.png]]

=====countRounds()=====
This function parses the rows of the review to determine how many rounds of reviews exist in the assignment. This allows us to intelligently print "Round 2 -- Question 3" for multi round reviews, or "Question 3" for single round reviews.

[[File:countRounds.png]]

=====getGridWidth()=====
This function parses the row data to determine how many tag prompts were used in this deployment using a simple selection search for the largest number of tags in a review. Since different tag deployments use different numbers of prompts, the rendering of the heatgrid needs to address this flexibility. Furthermore, the design of this function will allow for future flexibility of tag deployments, i.e., where different reviews in the same assignment or round may have different numbers of tags.

[[File:getGridWidth.png]]

=====drawTagGrid()=====
This is the master function for the rendering of the tag grid. This function sets up tooltip text, calls countRounds() and getGridWidth() and passes control to the three sub-functions, drawHeader(), drawQuestionRow(), and drawReviewRow(), depending on what type of item needs to be rendered next.

[[File:drawTagGrid.png]]

=====drawHeader()=====
This function draws the two-row header of the tag heatgrid which will contain "Tags Completed" and "15 out of 425". It also calls addToolTip() to add the tooltips for expand/contract, and the color legend for the tag fraction row.

[[File:drawHeader.png]]

=====drawQuestionRow()=====
This function is called once per question during the rendering process, and draws the row which will read, "Question 3" or "Round 2 -- Question 3". This also calls addToolTip() to add a color legend tooltip for the body of the heatgrid.

[[File:drawQuestionRow.png]]

=====drawReviewRow()=====
This function draws each row containing the presence, or absence of, tag prompts for each review. These contain R.# (Review.number), and a unicode symbol (set in the global variables). The unicode symbol is to enable accessibility for users who cannot see the red/green spectrum clearly, and will be populated with either a Universal No to mean no tags available, a Warning symbol, to mean tags not done, or a Check-Mark to indicate tags done.

[[File:drawReviewRow.png]]

Lastly, a call is made to updateTagsFraction() as part of the initial render to update the "12 out of 230" header row in the tag grid.

====Updating and Interaction Functions - view_team_in_grades.js====
=====tagActionOnUpdate()=====
Each time a tag prompt is changed, this function is called to update the heatgrid. This function calls several other functions, some of which have been used previously it the rendering step.

[[File:tagActionOnUpdate.png]]

=====updateTagsFraction()=====
This function is used both once in the rendering step, and again each time a tag prompt is changed. This function takes the data from calcTagRatio() and updates the contents and color of the header row in the tag heatgrid ("12 out of 220")

[[File:updateTagsFraction.png]]

=====updateTagGrid()=====
This function queries the tag prompts on the page, and uses these data to update the tag heatgrid class and text.

[[File:updateTagGrid.png]]

=====toggleHeatGridRows()=====
This function "Collapses" the heatgrid down to only the two header rows. Note that this function is called both onClick of the heatgrid header, so a user can collapse the grid manually, and automatically when the tag fraction is updated. This ensures that a user visiting the page with all tags complete will see a collapsed heatgrid by default.

[[File:toggleHeatGridRows.png]]

====Stylesheets - grades.scss====
We augmented the stylesheets used in grades so that the new heatgrid could be styled as small as possible to save page real estate. To do so, we added new classes in grades.scss

[[File:stylesheetsGrades_scss.png]]

===Client-Side Implementation: "Alternate View" (view_my_scores.html.erb)===
=====countTotalTags()=====
A simple JS function to count all the tag elements on current page. It will be invoked when the page first being rendered and also when any tags are changed by users.
[[File:countTotalTags.png]]

=====countTaggedTags()=====
A simple JS function to count the number of tagged tags on the current page. It will be invoked when the page first being rendered and when any tags are changed by users.
[[File:countTaggedTags.png]]

=====Additional Logic Check=====
This JS helper dynamically render the page when any tags are changed by users. Since we adopt different logic on two view pages, we added a condition check to tell which page we are on and apply the corresponding logic.
[[File:answer_tag.png]]

==Testing==
Our modifications were mostly on the front end of the code in the app/views and app/assets sections so we decided to create a spec/features test. Our feature test is named view_team_spec.rb and its purpose is to ensure that the table for the heatgrid is generated when accessing the "Your Scores" section. We were able to accomplish this by mostly drawing off of pre-existing feature tests namely peer_review_spec.rb, which had a user access and review another project.

You can run our created feature test with the following command in the expertiza directory.
rspec spec/features/view_team_spec.rb

Expected Output:
1 example, 0 failures

Since we did not make any changes to the model or controller classes we did construct any RSpec tests for any model or controller behavior.

===Navigating to Tag Report===
For manual testing purposes, we logged in as instructor6 and impersonated a student to view the reviews given to that student and be able to tag the reviews and see if the tagged reviews are updated in the heatmap.

===Manual Testing Instructions===
1. Login to Expertiza as "instructor6" with password "password". If hosted locally the url to access Expertiza should be "localhost:3000". 
2. Click on Manage... then "Impersonate User" 
3. Enter "student7856" into the field and click the Impersonate button 
You should see the following page. 
[[File:ImpersonateStudent.PNG]] 
4. Click on "Program 2" 
5. Click on "Your Scores" 
You should see the following page. 
[[File:YourScores.PNG]]
6. You should be able to click on a review score (1-5) and have the table expand, if you tag or remove a tag from a review, you should see the table on the right hand side of the page update to reflect your changes. 
7. Click Back and then click on "Alternate View" 
You should see the following page. 
[[File:AlternateView.PNG]]

===Testing HTML changes using Capybara===
rspec

==Additional Modifications -- Bugfix in Assignment Model and Grades Controller==
In developing the tagging report, the team noticed a significant, pre-existing view bug on view_team in the beta branch of Expertiza. This bug was causing rows of the review heatgrid to not be populated when the page was rendered. The team is still in communication with the main Expertiza development team to review and verify the validity of these modifications. At the time of this document, all build tests are passing.

'''Screenshot of unpatched bug: '''
[[File:unpatched_view_bug.png|1000px]]

'''Screenshot with patch applied: '''
[[File:patched_view_bug.png|1000px]]

===Origin of Bug===
We discovered a twofold cause of this bug. In part, in the '''grades controller''', calls were being made to an ActiveRecord query, '''@assignment.vary_by_round,''' which was always returning zero. Upon changing this query to use the helper method, '''varying_rubrics_by_round?''' from the '''Assignment''' model, we discovered that a recent change to the logic of '''varying_rubrics_by_round?''' was also causing unexpected behavior. We have included four modifications to our pull request that patch this bug.

===Patches to Assignment model===
In the '''varying_rubrics_by_round?''' method in the '''Assignment''' model, we discovered a recent change to the logic that was causing this method to test whether there were 2 or more '''AssignmentQuestionnaires''' assigned to Round 2 of reviews. Instead, this method should test whether there are 2 or more AssignmentQuestionnaires used on the assignment. We changed this logic to query using <code>used_in_round [1,2]</code> instead of <code>used_in_round 2</code>, which should evaluate '''.size > 1 TRUE''' if each round has a different '''AssignmentQuestionnaire'''.

[[File:assignment_model_patch.png]]

===Patches to Grades Controller===
In the grades controller, queries were being made to <code>@assignment.vary_by_round</code> instead of the helper method, <code>@assignment.varying_rubrics_by_round?</code>. These queries were always returning 0, because that DB field in our DB appears to be 0 for all table rows. We changed the 4 calls from this query to method calls.

[[File:grades_controller_patch.png]]

===Updates to Assignment Tests===
We also discovered an issue with how <code>varying_rubrics_by_round?</code> is tested. It seems, in the unmodified beta branch version, the test that should expect TRUE when rubrics are different for round 1 and 2, is actually testing whether Round 2 has more than 1 rubric assigned to it. We would expect this test to pass when using the old logic in <code>varying_rubrics_by_round?</code>. We modified the TRUE case to assign AssignmentQuestionnaire 1 to round 2, and AssignmentQuestionnaire 2 to round 1.

Also, it seemed that the test for FALSE was testing using one questionnaire in round 1, and a different questionnaire in round 2, which seems logically incorrect for this method. We believe the false case should test for the same rubric across two rounds, in which case the rubrics would not be varying by round. We modified the FALSE case to assign AssignmentQuestionnaire 1 to both round 1 and round 2.

[[File:assignment_spec_patch.png]]

===Updates to Grades Controller Tests===
Lastly, the changes to how <code>varying_rubrics_by_round?</code> is implemented meant that the two tests for the grades controller which rely on this method needed to be updated to expect the new logic. We implemented these updates to these two tests:

[[File:grades_controller_spec.png]]

File:YourScores.PNG

2021-03-19T20:39:26Z

Jmhurst:

File:ImpersonateStudent.PNG

2021-03-19T20:39:13Z

Jmhurst:

File:AlternateView.PNG

2021-03-19T20:38:53Z

Jmhurst: Jmhurst uploaded a new version of File:AlternateView.PNG