CSC/ECE 517 Spring 2017/finalproject E1744

CSC517 Final Project - E1744 Github Metrics

(asorgiu, george2, mdunlap, ygou14)

Proposed Design Document

Description

We will add a new feature to provide Expertiza with Github metrics (for example, number of committers, number of commits, number of lines of code modified, number of lines added, number of lines deleted.) from each group’s submitted repo link. This information should prove useful for differentiating the performance of team members for grading purposes. It may also help instructors to predict which projects are likely to be accepted/rejected (even before the final due dates).

Work to be done

This project is divided into two parts. One is to extract Github metadata of the submitted repos and pull requests. The second part is to build a classifier (e.g., Bayesian) to do the early prediction on some projects that are likely to fail. This prediction is based on more than 200 past projects (Fall 2012- Fall 2016). Based on the meta-data from students repos/pull requests, we can warn both authors and teaching staff if our model predicts that some projects are likely to fail.

The methodology of this project is to add a means to monitor the individual contributions of various team members throughout the duration of project in order to quantitatively access their work. This will aid the teaching staff and team members during the review process as well as improve visibility to a student of the work he or she has committed.

Extract Github metadata

• First, get an access token from github. Here are the steps
• Save the access token in the environment variable 'EXPERTIZA_GITHUB_TOKEN'
• Now, github data is fetched from github's Statistic API

Architectural Design

This feature has similar functionality with a web crawler, which is crawling the data from a server and store locally. So that for the architectural style of our subsystem, we would like to choose client/server style, which segregates the system into two applications, where the client makes requests to the server whenever a user is looking for the metrics. In many cases, the server is a database with application logic represented as stored procedures, in our case, is Github.

UML

Database Schema Changes

A new table called github_contributors is created to store the data for each committer. The table contain's the committer's email, github_id and all the metrics associated with a project. At the moment we handle the following metrics:

• Committer email - commiter_url
• Committer id - commiter_id
• Total number of commits - total_commits
• Number of files changed - files_changed
• Lines of code changed - lines_changed
• Lines of code added - lines_added
• Lines of code removed - lines_removed
• Lines of code added that survived until final submission - lines_persisted.

An index on committer_id is added to enable search.

A new table called submission_records_github_contributors which acts as a reference between the submission_records and github_contributors tables. It has two columns:

• github_contributor_id - Foreign Key to github_contributors table.
• submission_record_id - Foreign Key to submission_records table.

A composite unique key constraint is added on github_contributor_id and submission_record_id.

Test Plan

The tests will use rspec to validate the response to the metrics from github and displaying the various metrics. These tests will be test the data in new tables submission_records_github_contributors and github_contributors as well as expand the existing tests on submission_records.

1. Test 1:
   1. Login as instructor6
   2. Go to submission page
   3. Validate that metrics are pulled in from github
2. Test 2:
   1. Login as student
   2. Go to submission page
   3. Confirm the student's metrics values

Unit Tests

Unit Test Summary
Method	Parameter	Expected result
show	submission_id with valid github hyperlink	Github Metrics and status code 200
show	submission_id with file uploaded	render 'github_contributors/not_found'
show	submission_id with non github hyperlink	render 'github_contributors/not_found'
show	submission_id with private github hyperlink	render 'github_contributors/not_found'

Build a classifier

THIS WILL NOT BE IMPLEMENTED AS PART OF THIS PROJECT. This is future work to be done.