CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

Introduction

Problem Statement

Expertiza provides Teammate Reviews under View Scores functionality for each assignment. Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub. This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

1. Teammate Reviews functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

2. Github Metrics under View Submissions page should include a bar chart that shows number of commits by the team throughout the assignment timeline. This will help instructors to get a team overview, and aid grading process.

While this data will not have marks associated directly, it will prove useful to the instructor in differentiating the performance of team members and hence awarding marks as per contribution. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

Current Scenario

At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.

Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

Implemented Solution

Files Modified

• app/controllers/auth_controller.rb
• app/controllers/grades_controller.rb
• app/helpers/grades_helper.rb
• app/views/assignments/list_submissions.html.erb
• app/views/grades/_tabbing.html.erb
• app/views/grades/_teammate_reviews_tab.html.erb
• app/views/grades/view.html.erb
• app/views/grades/view_team.html.erb
• config/application.rb
• config/initializers/load_config.rb
• config/initializers/omniauth.rb
• config/routes.rb

Files Added

• app/views/grades/view_github_metrics.html.erb
• config/github_auth.yml

First Change

• A new table "Github Metrics" is added under Manage-> Assignments -> View Scores -> Teammate Reviews. Below is the screenshot of the implementation.

The GitHub metrics table shows results for each team member

Second Change

• The second change is in the View Submissions page, where we have added a link "Github Metrics" to a new page.

The GitHub metrics link can be seen below Assign Grade link

• The new page appears after clicking on the link "Github metrics", that shows bar chart for # of commits per day. We have also added other relevant information about Pull Request, such as total commits, lines of code added, lines of code modified, PR merge status, check status.

The bar chart

Test Plan

Subtask 1: GitHub metrics in teammate reviews

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data per team member based on different GitHub metrics (e.g. lines of code added/changed/removed etc.)

Subtask 2: Line chart for # of lines changed by the overall team

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Click on the "Github metrics" link for the team whose metrics you wish to view

5) A new page opens and shows # of lines changed per team member since the start of the assignment

Change-log for Reviewers

This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.

• Removed "View submission" changes as per updated requirements (by the Professor).
• Updated "Introduction" to explain how the Github data will affect grading.
• Updated "Design Considerations" with results of further investigation and our conclusions as to which approach we will exactly take.
• Updated "Plan of Action" section to include which files and functions will be exactly updated.
• Updated "Proposed Solution and test plan" section to include modified views after discussing with the mentor.

References

Expertiza_wiki

E1815:_Improvements_to_review_grader

Expertiza_PR_1179

Expertiza_PR_1179_Video

GitHub API documentation