CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration
Introduction
Problem Statement
Expertiza provides Teammate Reviews under View Scores functionality for each assignment. Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub. This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.
- 1. Teammate Reviews functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.
- 2. Github Metrics under View Submissions page should include a bar chart that shows number of commits by the team throughout the assignment timeline. This will help instructors to get a team overview, and aid grading process.
While this data will not have marks associated directly, it will prove useful to the instructor in differentiating the performance of team members and hence awarding marks as per contribution. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.
Current Scenario
Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.
Use Case Diagram
Use Case Diagram Details
Actors:
- Instructor: This actor is responsible for viewing GitHub metrics of teams and team members of an assignment.
Pre-Conditions:
- The Team should have submitted the assignment with a PR link or GitHub repository.
Primary Sequence:
- The instructor should login.
- The instructor should browse teams for an assignment.
Post Conditions:
- Instructor will be able to see the team contribution done by each team member in 'View Submissions' page using graph diagrams, as shown in the figure.
- Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metrics table appended at the end, as shown in the figure.
Design Principles
- MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.
- Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.
Solution Design
- The Github metrics that need to be integrated with Expertiza were finalized as below. These metrics are captured on a per-user basis:
- Total number of commits.
- Number of files changed.
- Lines of Code added
- Lines of code modified.
- Lines of code deleted.
- Pull Request Status ( includes code climate and Travis CI Build status)
- User Github metrics:
- Committer ID
- Committer Name
- Committer email ID
- A new link "Github Metrics" is provided under “View Submissions” for an assignment in the instructor view.This link opens a new tab and shows a stacked bar chart for number of commits per user vs submission timeline from assignment creation date to the deadline.
- In "View Scores" for an assignment in the instructor view, under Teammate Reviews tab, a new table for Github Metrics is added, which shows following Github metrics per user:
- Student Name/ID, Email ID, lines of code added, lines of code deleted, number of commits
- For GitHub integration, we have used GitHub GraphQL API v4. We have used OAuth gem for authentication purpose.
- We parse the link to PR to get data associated with it. We have also handled projects which do not have PR link, but just a link to the repository.
Implemented Solution
Files Modified
- app/controllers/auth_controller.rb
- app/controllers/grades_controller.rb
- app/helpers/grades_helper.rb
- app/views/assignments/list_submissions.html.erb
- app/views/grades/_tabbing.html.erb
- app/views/grades/_teammate_reviews_tab.html.erb
- app/views/grades/view.html.erb
- app/views/grades/view_team.html.erb
- config/application.rb
- config/initializers/load_config.rb
- config/initializers/omniauth.rb
- config/routes.rb
Files Added
- app/views/grades/view_github_metrics.html.erb
- config/github_auth.yml
First Change
- A new table "Github Metrics" is added under Manage-> Assignments -> View Scores -> Teammate Reviews. Below is the screenshot of the implementation.
Second Change
- The second change is in the View Submissions page, where we have added a link "Github Metrics" to a new page.
- The new page appears after clicking on the link "Github metrics", that shows bar chart for # of commits per day. We have also added other relevant information about Pull Request, such as total commits, lines of code added, lines of code modified, PR merge status, check status.
Code Change in Grades_Controller
- Added below new functions to implement Github Integration in View Submission page
def get_statuses_for_pull_request(ref)
url = "https://api.github.com/repos/expertiza/expertiza/commits/" + ref + "/status"
ActiveSupport::JSON.decode(Net::HTTP.get(URI(url)))
end
def retrieve_pull_request_data(pull_links)
pull_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["pull_request_number"] = submission_hyperlink_tokens.pop
submission_hyperlink_tokens.pop
hyperlink_data["repository_name"] = submission_hyperlink_tokens.pop
hyperlink_data["owner_name"] = submission_hyperlink_tokens.pop
github_data = get_pull_request_details(hyperlink_data)
parse_github_data_pull(github_data)
end
end
def retrieve_repository_data(repo_links)
repo_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["repository_name"] = submission_hyperlink_tokens[4]
next if hyperlink_data["repository_name"] == "servo" || hyperlink_data["repository_name"] == "expertiza"
hyperlink_data["owner_name"] = submission_hyperlink_tokens[3]
github_data = get_github_data_repo(hyperlink_data)
parse_github_data_repo(github_data)
end
end
def retrieve_github_data
team_links = @team.hyperlinks
pull_links = team_links.select do |link|
link.match(/pull/) && link.match(/github.com/)
end
if !pull_links.empty?
retrieve_pull_request_data(pull_links)
else
repo_links = team_links.select do |link|
link.match(/github.com/)
end
retrieve_repository_data(repo_links)
end
end
def retrieve_check_run_statuses
@head_refs.each do |pull_number, ref|
@check_statuses[pull_number] = get_statuses_for_pull_request(ref)
end
end
def view_github_metrics
if session["github_access_token"].nil?
session["participant_id"] = params[:id]
session["github_view_type"] = "view_submissions"
redirect_to authorize_github_grades_path
return
end
@head_refs = {}
@parsed_data = {}
@authors = {}
@dates = {}
@total_additions = 0
@total_deletions = 0
@total_commits = 0
@total_files_changed = 0
@merge_status = {}
@check_statuses = {}
@token = session["github_access_token"]
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
retrieve_github_data
retrieve_check_run_statuses
@authors = @authors.keys
@dates = @dates.keys.sort
end
def authorize_github
redirect_to "https://github.com/login/oauth/authorize?client_id=#{GITHUB_CONFIG['client_key']}"
end
def get_github_data_repo(hyperlink_data)
data = {
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name: \"" + hyperlink_data["repository_name"] + "\") {
ref(qualifiedName: \"master\") {
target {
... on Commit {
id
history(first: 100) {
edges {
node {
id author {
name email date
}
}
}
}
}
}
}
}
}"
}
make_github_api_request(data)
end
def get_pull_request_details(hyperlink_data)
response_data = make_github_api_request(get_query(true, hyperlink_data))
@has_next_page = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]
while @has_next_page
new_response_data = make_github_api_request(get_query(false, hyperlink_data))
response_data["data"]["repository"]["pullRequest"]["commits"]["edges"].push(*new_response_data["data"]["repository"]["pullRequest"]["commits"]["edges"])
@has_next_page = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]
end
response_data
end
def process_github_authors_and_dates(author_name, commit_date)
@authors[author_name] ||= 1
@dates[commit_date] ||= 1
@parsed_data[author_name] ||= {}
@parsed_data[author_name][commit_date] = if @parsed_data[author_name][commit_date]
@parsed_data[author_name][commit_date] + 1
else
1
end
end
def parse_github_data_pull(github_data)
team_statistics(github_data)
pull_request_object = github_data["data"]["repository"]["pullRequest"]
commit_objects = pull_request_object["commits"]["edges"]
commit_objects.each do |commit_object|
commit = commit_object["node"]["commit"]
author_name = commit["author"]["name"]
commit_date = commit["committedDate"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end
def parse_github_data_repo(github_data)
commit_history = github_data["data"]["repository"]["ref"]["target"]["history"]
commit_objects = commit_history["edges"]
commit_objects.each do |commit_object|
commit_author = commit_object["node"]["author"]
author_name = commit_author["name"]
commit_date = commit_author["date"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end
def make_github_api_request(data)
uri = URI.parse("https://api.github.com/graphql")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
request = Net::HTTP::Post.new(uri.path, 'Authorization' => 'Bearer' + ' ' + session["github_access_token"])
request.body = data.to_json
http.request(request)
response = http.request(request)
ActiveSupport::JSON.decode(response.body.to_s)
end
def organize_commit_dates
@dates.each_key do |date|
@parsed_data.each_value do |commits|
commits[date] ||= 0
end
end
@parsed_data.each {|author, commits| @parsed_data[author] = Hash[commits.sort_by {|date, _commit_count| date }] }
end
def team_statistics(github_data)
@total_additions += github_data["data"]["repository"]["pullRequest"]["additions"]
@total_deletions += github_data["data"]["repository"]["pullRequest"]["deletions"]
@total_files_changed += github_data["data"]["repository"]["pullRequest"]["changedFiles"]
@total_commits += github_data["data"]["repository"]["pullRequest"]["commits"]["totalCount"]
pull_request_number = github_data["data"]["repository"]["pullRequest"]["number"]
@head_refs[pull_request_number] = github_data["data"]["repository"]["pullRequest"]["headRefOid"]
@merge_status[pull_request_number] = if github_data["data"]["repository"]["pullRequest"]["merged"]
"MERGED"
else
github_data["data"]["repository"]["pullRequest"]["mergeable"]
end
end
def get_query(is_initial_page, hyperlink_data)
commit_query_line = if is_initial_page
"commits(first:100){"
else
"commits(first:100, after:" + @end_cursor + "){"
end
{
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name:\"" + hyperlink_data["repository_name"] + "\") {
pullRequest(number: " + hyperlink_data["pull_request_number"] + ") {
number additions deletions changedFiles mergeable merged headRefOid
" + commit_query_line + "
totalCount
pageInfo{
hasNextPage startCursor endCursor
}
edges{
node{
id commit{
author{
name
}
additions deletions changedFiles committedDate
}}}}}}}"
}
end
Test Plan
Change 1: GitHub metrics in teammate reviews
1) Log in as an instructor (instructor6/password)
2) Navigate to assignments through Manage --> Assignments
3) Select "View scores" icon for the assignment of your choice
4) Select the team for which you wish to view scores
5) Go to "Teammate Reviews" tab
6) View data per team member based on different GitHub metrics (e.g. lines of code added/changed/removed etc.)
Change 2: Bar chart for # of commits changed by the overall team
1) Log in as an instructor (instructor6/password)
2) Navigate to assignments through Manage --> Assignments
3) Select "View submissions" icon for the assignment of your choice
4) Click on the "Github metrics" link for the team whose metrics you wish to view
5) A new page opens and shows # of commits changed per team member since the start of the assignment, also bottom of the page shows summary from Github submissions.
RSpec Tests
- Following feature tests were added to the Grades_Controller_Spec.rb
describe '#get_statuses_for_pull_request' do
before(:each) do
allow(Net::HTTP).to receive(:get) {"{\"team\":\"rails\",\"players\":\"36\"}"}
end
it 'makes a call to the GitHub API to get status of the head commit passed' do
expect(controller.get_statuses_for_pull_request('qwerty123')).to eq({"team" => "rails", "players" => "36"})
end
end
describe '#retrieve_pull_request_data' do
before(:each) do
allow(controller).to receive(:get_pull_request_details).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_pull)
end
it 'gets pull request details for each PR link submitted' do
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1261",
"repository_name" => "expertiza",
"owner_name" => "expertiza"
})
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1293",
"repository_name" => "mamaMiya",
"owner_name" => "Shantanu"
})
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end
it 'calls parse_github_data_pull on each of the PR details' do
expect(controller).to receive(:parse_github_data_pull).with({"pr" => "details"}).twice
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end
end
describe '#retrieve_repository_data' do
before(:each) do
allow(controller).to receive(:get_github_data_repo).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_repo)
end
it 'gets details for each repo link submitted, excluding those for expertiza and servo' do
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "website",
"owner_name" => "Shantanu"
})
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "OODD",
"owner_name" => "Edward"
})
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD", "https://github.com/expertiza/expertiza", "https://github.com/Shantanu/expertiza]"])
end
it 'calls parse_github_data_repo on each of the PR details' do
expect(controller).to receive(:parse_github_data_repo).with({"pr" => "details"}).twice
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD"])
end
end
describe '#retrieve_github_data' do
before(:each) do
allow(controller).to receive(:retrieve_pull_request_data)
allow(controller).to receive(:retrieve_repository_data)
end
context 'when pull request links have been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/Shantanu/website/pull/1123"])
controller.instance_variable_set(:@team, teams_mock)
end
it 'retrieves PR data only' do
expect(controller).to receive(:retrieve_pull_request_data).with(["https://github.com/Shantanu/website/pull/1123"])
controller.retrieve_github_data
end
end
context 'when pull request links have not been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.instance_variable_set(:@team, teams_mock)
end
it 'retrieves repo details ' do
expect(controller).to receive(:retrieve_repository_data).with(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.retrieve_github_data
end
end
end
describe '#retrieve_check_run_statuses' do
before(:each) do
allow(controller).to receive(:get_statuses_for_pull_request).and_return("check_status")
controller.instance_variable_set(:@headRefs, {"1234" => "qwerty", "5678" => "asdfg"})
controller.instance_variable_set(:@check_statuses, {})
end
it 'gets and stores the statuses associated with head commits of PRs' do
expect(controller).to receive(:get_statuses_for_pull_request).with("qwerty")
expect(controller).to receive(:get_statuses_for_pull_request).with("asdfg")
controller.retrieve_check_run_statuses
expect(controller.instance_variable_get(:@check_statuses)).to eq({"1234" => "check_status", "5678" => "check_status"})
end
end
describe '#view_github_metrics' do
context 'when user hasn\'t logged in to GitHub' do
before(:each) do
@params = {id: 900}
session["github_access_token"] = nil
end
it 'stores the current participant id and the view action' do
get :view_github_metrics, @params
expect(session["participant_id"]).to eq("900")
expect(session["github_view_type"]).to eq("view_submissions")
end
it 'redirects user to GitHub authorization page' do
get :view_github_metrics, @params
expect(response).to redirect_to(authorize_github_grades_path)
end
end
context 'when user has logged in to GitHub' do
before(:each) do
session["github_access_token"] = "qwerty"
allow(controller).to receive(:get_statuses_for_pull_request).and_return("status")
allow(controller).to receive(:retrieve_github_data)
allow(controller).to receive(:retrieve_check_run_statuses)
end
it 'stores the GitHub access token for later use' do
get :view_github_metrics, {id: '1'}
expect(controller.instance_variable_get(:@token)).to eq("qwerty")
end
it 'calls retrieve_github_data to retrieve data from GitHub' do
expect(controller).to receive(:retrieve_github_data)
get :view_github_metrics, {id: '1'}
end
it 'calls retrieve_check_run_statuses to retrieve check runs data' do
expect(controller).to receive(:retrieve_check_run_statuses)
get :view_github_metrics, {id: '1'}
end
end
end
end
Change-log for Reviewers
This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.
- Added Solution Design for the final implemented design
- Added Implemented Solution to show feature additions to Expertiza
- Added Feature test cases for Grades controller