CSC/ECE 517 Spring 2017/finalproject E1744

From Expertiza_Wiki
Jump to navigation Jump to search

CSC517 Final Project - E1744 Github Metrics

(asorgiu, george2, mdunlap, ygou14)


Proposed Design Document

Description

We will add a new feature to provide Expertiza with Github metrics (for example, number of committers, number of commits, number of lines of code modified, number of lines added, number of lines deleted.) from each group’s submitted repo link. This information should prove useful for differentiating the performance of team members for grading purposes. It may also help instructors to predict which projects are likely to be accepted/rejected (even before the final due dates).

Work to be done

This project is divided into two parts. One is to extract Github metadata of the submitted repos and pull requests. The second part is to build a classifier (e.g., Bayesian) to do the early prediction on some projects that are likely to fail. This prediction is based on more than 200 past projects (Fall 2012- Fall 2016). Based on the meta-data from students repos/pull requests, we can warn both authors and teaching staff if our model predicts that some projects are likely to fail.

The methodology of this project is to add a means to monitor the individual contributions of various team members throughout the duration of project in order to quantitatively access their work. This will aid the teaching staff and team members during the review process as well as improve visibility to a student of the work he or she has committed.

Extract Github metadata
  • First, get an access token from github. Here are the steps
  • Save the access token in the environment variable 'EXPERTIZA_GITHUB_TOKEN'
  • Now, github data is fetched from github's Statistic API
Architectural Design

This feature has similar functionality with a web crawler, which is crawling the data from a server and store locally. So that for the architectural style of our subsystem, we would like to choose client/server style, which segregates the system into two applications, where the client makes requests to the server whenever a user is looking for the metrics. In many cases, the server is a database with application logic represented as stored procedures, in our case, is Github.

UML
Database Schema Changes

A new table called github_contributors is created to store the data for each committer. The table contain's the committer's email, github_id and all the metrics associated with a project. At the moment we handle the following metrics:

  • Committer email - commiter_url
  • Committer id - commiter_id
  • Total number of commits - total_commits
  • Number of files changed - files_changed
  • Lines of code changed - lines_changed
  • Lines of code added - lines_added
  • Lines of code removed - lines_removed
  • Lines of code added that survived until final submission - lines_persisted.

An index on committer_id is added to enable search.

A new table called submission_records_github_contributors which acts as a reference between the submission_records and github_contributors tables. It has two columns:

  • github_contributor_id - Foreign Key to github_contributors table.
  • submission_record_id - Foreign Key to submission_records table.

A composite unique key constraint is added on github_contributor_id and submission_record_id.

Test Plan

The tests will use rspec to validate the response to the metrics from github and displaying the various metrics. These tests will be test the data in new tables submission_records_github_contributors and github_contributors as well as expand the existing tests on submission_records.

  1. Test 1:
    1. Login as instructor6
    2. Go to submission page
    3. Validate that metrics are pulled in from github
  2. Test 2:
    1. Login as student
    2. Go to submission page
    3. Confirm the student's metrics values
Unit Tests
Unit Test Summary
Method Parameter Expected result
get_metrics team_id = null Null / Exception
get_metrics team_id = Invalid team_id Null / Exception
get_metrics team_id = valid_team_id List of metrics for all the committers in the team.

Build a classifier

THIS WILL NOT BE IMPLEMENTED AS PART OF THIS PROJECT. This is future work to be done.

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2017/finalproject_E1744&oldid=108660"