CSC/ECE 517 Spring 2017/finalproject E1744

CSC517 Final Project - E1744 Github Metrics

(asorgiu, george2, mdunlap, ygou14)

Proposed Design Document

Description

We will add a new feature to provide Expertiza with Github metrics (for example, number of committers, number of commits, number of lines of code modified, number of lines added, number of lines deleted.) from each group’s submitted repo link. This information should prove useful for differentiating the performance of team members for grading purposes. It may also help instructors to predict which projects are likely to be merged.

Work to be done

This project is divided into two parts. One is to extract Github metadata of the submitted repos and pull requests. The second part is to build a classifier (e.g., Bayesian) to do the early prediction on some projects that are likely to fail. This prediction is based on more than 200 past projects. The features above should be used, together with some temporal features (e.g. the temporal pattern of this team’s commits so far). Eventually, we would like to e-mail students whose metrics are bad, giving them advice on how to improve.

The purpose of this project is to add a means to monitor the individual contributions of various team members throughout the duration of project in order to quantitatively access their work. This will aid the teaching staff and team members during the review process as well as improve visibility to a student of the work he or she has committed.

Extract Github metadata

Data Flow

The code should sync the data with Github whenever someone (student or instructor) looks at a view that shows Github data.

Architectural Design

This feature has similar functionality with a web crawler, which is crawling the data from a server and store locally. So that for the architectural style of our subsystem, we would like to choose client/server style, which segregates the system into two applications, where the client makes requests to the server whenever a user is looking for the metrics. In many cases, the server is a database with application logic represented as stored procedures, in our case, is Github.

UML

Database Schema Changes

A new table called github_contributors is created to store the data for each committer. The table contain's the committer's email, github_id and all the metrics associated with a project. At the moment we handle the following metrics:

• Committer email - commiter_url
• Committer id - commiter_id
• Total number of commits - total_commits
• Number of files changed - files_changed
• Lines of code changed - lines_changed
• Lines of code added - lines_added
• Lines of code removed - lines_removed
• Lines of code added that survived until final submission - lines_persisted.

An index on committer_id is added to enable search.

A new table called submission_records_github_contributors which acts as a reference between the submission_records and github_contributors tables. It has two columns:

• github_contributor_id - Foreign Key to github_contributors table.
• submission_record_id - Foreign Key to submission_records table.

A composite unique key constraint is added on github_contributor_id and submission_record_id.

Test Plan

The tests will use rspec to validate the response to the metrics from github and displaying the various metrics. These tests will be test the data in new tables submission_records_github_contributors and github_contributors as well as expand the existing tests on submission_records.

1. Test 1:
   1. Login as instructor6
   2. Go to submission page
   3. Validate that metrics are pulled in from github
2. Test 2:
   1. Login as student
   2. Go to submission page
   3. Confirm the student's metrics values

Unit Tests

Unit Test Summary
Method	Parameter	Expected result
get_metrics	team_id = null	Null / Exception
get_metrics	team_id = Invalid team_id	Null / Exception
get_metrics	team_id = valid_team_id	List of metrics for all the committers in the team.

Build a classifier

THIS WILL NOT BE IMPLEMENTED AS PART OF THIS PROJECT. This is future work to be done.