Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-12-16T03:45:54Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''Teammate Reviews''' under '''View Scores''' functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.
:2. '''Github Metrics''' under '''View Submissions''' page should include a bar chart that shows number of commits by the team throughout the assignment timeline. This will help instructors to get a team overview, and aid grading process.

While this data will not have marks associated directly, it will prove useful to the instructor in differentiating the performance of team members and hence awarding marks as per contribution. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
Previously, view submission does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

==='''Use Case Diagram'''===
[[File:UC_1.png|none|frame|1. Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. 2. Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for viewing GitHub metrics of teams and team members of an assignment.

'''Pre-Conditions:'''

*The Team should have submitted the assignment with a PR link or GitHub repository.

'''Primary Sequence:'''

*The instructor should login.

*The instructor should browse teams for an assignment.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submissions' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metrics table appended at the end, as shown in the figure.

=='''Design Considerations'''==

* The first thing was to determine what metrics we are looking for exactly. These are what the solution supports:
*# Number of commits per user and total per team.
*# Lines of Code added
*# Lines of code deleted.

* The next thing was to narrow down what hosting service for version control we would use. For now, we only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately (changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with (people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
* We also considered if we needed to account for different branches. We only consider the master branch.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories (like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Solution Design'''==

* The Github metrics that need to be integrated with Expertiza were finalized as below. These metrics are captured on a per-user basis:
*# Total number of commits.
*# Lines of Code added
*# Lines of code deleted.
*# Pull Request Status ( includes code climate and Travis CI Build status)
*#User Github metrics:
*## Committer ID
*## Committer Name
*## Committer email ID

* A new link "Github Metrics" is provided under “View Submissions” for an assignment in the instructor view.This link opens a new tab and shows a stacked bar chart for number of commits per user vs submission timeline from assignment creation date to the deadline.
* In "View Scores" for an assignment in the instructor view, under Teammate Reviews tab, a new table for Github Metrics is added, which shows following Github metrics per user:
:Student Name/ID, Email ID, lines of code added, lines of code deleted, number of commits

* For GitHub integration, we have used GitHub GraphQL API v4. We have used github-omniauth gem for authentication/authorization purposes.

* We parse the link to PR to get data associated with it. We have also handled projects which do not have PR link, but just a link to the repository. We excluded expertiza and servo projects as right now a PR link is expected. Future enhancements can look into getting separate GitHub submission links.

* We also show the status of check runs in the View Github metrics view to help instructors view the status of various tools on the repos/PRs without having to go to the actual GitHub page.

=='''Implemented Solution'''==

==='''Files Modified'''===

* app/controllers/auth_controller.rb
* app/controllers/grades_controller.rb
* app/helpers/grades_helper.rb
* app/views/assignments/list_submissions.html.erb
* app/views/grades/_tabbing.html.erb
* app/views/grades/_teammate_reviews_tab.html.erb
* app/views/grades/view.html.erb
* app/views/grades/view_team.html.erb
* config/application.rb
* config/initializers/load_config.rb
* config/initializers/omniauth.rb
* config/routes.rb

==='''Files Added'''===

* app/views/grades/view_github_metrics.html.erb
* config/github_auth.yml

==='''First Change'''===

* A new table "Github Metrics" is added under Manage-> Assignments -> View Scores -> Teammate Reviews. Below is the screenshot of the implementation.

[[File:Ooddafter4.png|none|frame|The GitHub metrics table shows results for each team member]]

==='''Second Change'''===
* The second change is in the View Submissions page, where we have added a link "Github Metrics" to a new page.

[[File:Ooddafter3.png|none|frame|
At present, view submission shows group assignments are submitted as a single submission and 'view github metric' link shows work contribution per teammate.
]]

* The new page appears after clicking on the link "Github metrics", that shows bar chart for # of commits per day. We have also added other relevant information about Pull Request, such as total commits, lines of code added, lines of code modified, PR merge status, check status.

[[File:Capture_3.png|none|frame|The bar chart]]
[[File:Ooddafter10.png|none|frame|The Github summary]]

===='''Code Change in Grades_Controller'''====
* Added below new functions to implement Github Integration in View Submission page
<pre>

def get_statuses_for_pull_request(ref)
url = "https://api.github.com/repos/expertiza/expertiza/commits/" + ref + "/status"
ActiveSupport::JSON.decode(Net::HTTP.get(URI(url)))
end

def retrieve_pull_request_data(pull_links)
pull_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["pull_request_number"] = submission_hyperlink_tokens.pop
submission_hyperlink_tokens.pop
hyperlink_data["repository_name"] = submission_hyperlink_tokens.pop
hyperlink_data["owner_name"] = submission_hyperlink_tokens.pop
github_data = get_pull_request_details(hyperlink_data)
parse_github_data_pull(github_data)
end
end

def retrieve_repository_data(repo_links)
repo_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["repository_name"] = submission_hyperlink_tokens[4]
next if hyperlink_data["repository_name"] == "servo" || hyperlink_data["repository_name"] == "expertiza"
hyperlink_data["owner_name"] = submission_hyperlink_tokens[3]
github_data = get_github_data_repo(hyperlink_data)
parse_github_data_repo(github_data)
end
end

def retrieve_github_data
team_links = @team.hyperlinks
pull_links = team_links.select do |link|
link.match(/pull/) && link.match(/github.com/)
end
if !pull_links.empty?
retrieve_pull_request_data(pull_links)
else
repo_links = team_links.select do |link|
link.match(/github.com/)
end
retrieve_repository_data(repo_links)
end
end

def retrieve_check_run_statuses
@head_refs.each do |pull_number, ref|
@check_statuses[pull_number] = get_statuses_for_pull_request(ref)
end
end

def view_github_metrics
if session["github_access_token"].nil?
session["participant_id"] = params[:id]
session["github_view_type"] = "view_submissions"
redirect_to authorize_github_grades_path
return
end

@head_refs = {}
@parsed_data = {}
@authors = {}
@dates = {}
@total_additions = 0
@total_deletions = 0
@total_commits = 0
@total_files_changed = 0
@merge_status = {}
@check_statuses = {}

@token = session["github_access_token"]

@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id

retrieve_github_data
retrieve_check_run_statuses

@authors = @authors.keys
@dates = @dates.keys.sort
end

def authorize_github
redirect_to "https://github.com/login/oauth/authorize?client_id=#{GITHUB_CONFIG['client_key']}"
end

def get_github_data_repo(hyperlink_data)
data = {
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name: \"" + hyperlink_data["repository_name"] + "\") {
ref(qualifiedName: \"master\") {
target {
... on Commit {
id
history(first: 100) {
edges {
node {
id author {
name email date
}
}
}
}
}
}
}
}
}"
}
make_github_api_request(data)
end

def get_pull_request_details(hyperlink_data)
response_data = make_github_api_request(get_query(true, hyperlink_data))

@has_next_page = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]

while @has_next_page
new_response_data = make_github_api_request(get_query(false, hyperlink_data))
response_data["data"]["repository"]["pullRequest"]["commits"]["edges"].push(*new_response_data["data"]["repository"]["pullRequest"]["commits"]["edges"])
@has_next_page = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]
end

response_data
end

def process_github_authors_and_dates(author_name, commit_date)
@authors[author_name] ||= 1
@dates[commit_date] ||= 1
@parsed_data[author_name] ||= {}
@parsed_data[author_name][commit_date] = if @parsed_data[author_name][commit_date]
@parsed_data[author_name][commit_date] + 1
else
1
end
end

def parse_github_data_pull(github_data)
team_statistics(github_data)
pull_request_object = github_data["data"]["repository"]["pullRequest"]
commit_objects = pull_request_object["commits"]["edges"]
commit_objects.each do |commit_object|
commit = commit_object["node"]["commit"]
author_name = commit["author"]["name"]
commit_date = commit["committedDate"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end

def parse_github_data_repo(github_data)
commit_history = github_data["data"]["repository"]["ref"]["target"]["history"]
commit_objects = commit_history["edges"]
commit_objects.each do |commit_object|
commit_author = commit_object["node"]["author"]
author_name = commit_author["name"]
commit_date = commit_author["date"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end

def make_github_api_request(data)
uri = URI.parse("https://api.github.com/graphql")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
request = Net::HTTP::Post.new(uri.path, 'Authorization' => 'Bearer' + ' ' + session["github_access_token"])
request.body = data.to_json
http.request(request)
response = http.request(request)
ActiveSupport::JSON.decode(response.body.to_s)
end

def organize_commit_dates
@dates.each_key do |date|
@parsed_data.each_value do |commits|
commits[date] ||= 0
end
end
@parsed_data.each {|author, commits| @parsed_data[author] = Hash[commits.sort_by {|date, _commit_count| date }] }
end

def team_statistics(github_data)
@total_additions += github_data["data"]["repository"]["pullRequest"]["additions"]
@total_deletions += github_data["data"]["repository"]["pullRequest"]["deletions"]
@total_files_changed += github_data["data"]["repository"]["pullRequest"]["changedFiles"]
@total_commits += github_data["data"]["repository"]["pullRequest"]["commits"]["totalCount"]
pull_request_number = github_data["data"]["repository"]["pullRequest"]["number"]
@head_refs[pull_request_number] = github_data["data"]["repository"]["pullRequest"]["headRefOid"]

@merge_status[pull_request_number] = if github_data["data"]["repository"]["pullRequest"]["merged"]
"MERGED"
else
github_data["data"]["repository"]["pullRequest"]["mergeable"]
end
end

def get_query(is_initial_page, hyperlink_data)
commit_query_line = if is_initial_page
"commits(first:100){"
else
"commits(first:100, after:" + @end_cursor + "){"
end
{
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name:\"" + hyperlink_data["repository_name"] + "\") {
pullRequest(number: " + hyperlink_data["pull_request_number"] + ") {
number additions deletions changedFiles mergeable merged headRefOid
" + commit_query_line + "
totalCount
pageInfo{
hasNextPage startCursor endCursor
}
edges{
node{
id commit{
author{
name
}
additions deletions changedFiles committedDate
}}}}}}}"
}
end

</pre>

=='''Test Plan'''==

'''Change 1: GitHub metrics in teammate reviews'''

1) Log in as an instructor (instructor6/password)

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate Reviews" tab

6) View data per team member based on different GitHub metrics (e.g. lines of code added/changed/removed etc.)

'''Change 2: Bar chart for # of commits changed by the overall team'''

1) Log in as an instructor (instructor6/password)

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Click on the "Github metrics" link for the team whose metrics you wish to view

5) A new page opens and shows # of commits changed per team member since the start of the assignment, also bottom of the page shows summary from Github submissions.

==='''RSpec Tests'''===

* Following feature tests were added to the Grades_Controller_Spec.rb
<pre>

describe '#get_statuses_for_pull_request' do
before(:each) do
allow(Net::HTTP).to receive(:get) {"{\"team\":\"rails\",\"players\":\"36\"}"}
end

it 'makes a call to the GitHub API to get status of the head commit passed' do
expect(controller.get_statuses_for_pull_request('qwerty123')).to eq({"team" => "rails", "players" => "36"})
end
end

describe '#retrieve_pull_request_data' do
before(:each) do
allow(controller).to receive(:get_pull_request_details).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_pull)
end

it 'gets pull request details for each PR link submitted' do
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1261",
"repository_name" => "expertiza",
"owner_name" => "expertiza"
})
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1293",
"repository_name" => "mamaMiya",
"owner_name" => "Shantanu"
})
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end

it 'calls parse_github_data_pull on each of the PR details' do
expect(controller).to receive(:parse_github_data_pull).with({"pr" => "details"}).twice
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end
end

describe '#retrieve_repository_data' do
before(:each) do
allow(controller).to receive(:get_github_data_repo).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_repo)
end

it 'gets details for each repo link submitted, excluding those for expertiza and servo' do
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "website",
"owner_name" => "Shantanu"
})
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "OODD",
"owner_name" => "Edward"
})
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD", "https://github.com/expertiza/expertiza", "https://github.com/Shantanu/expertiza]"])
end

it 'calls parse_github_data_repo on each of the PR details' do
expect(controller).to receive(:parse_github_data_repo).with({"pr" => "details"}).twice
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD"])
end
end

describe '#retrieve_github_data' do
before(:each) do
allow(controller).to receive(:retrieve_pull_request_data)
allow(controller).to receive(:retrieve_repository_data)
end

context 'when pull request links have been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/Shantanu/website/pull/1123"])
controller.instance_variable_set(:@team, teams_mock)
end

it 'retrieves PR data only' do
expect(controller).to receive(:retrieve_pull_request_data).with(["https://github.com/Shantanu/website/pull/1123"])
controller.retrieve_github_data
end
end

context 'when pull request links have not been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.instance_variable_set(:@team, teams_mock)
end

it 'retrieves repo details ' do
expect(controller).to receive(:retrieve_repository_data).with(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.retrieve_github_data
end
end
end

describe '#retrieve_check_run_statuses' do
before(:each) do
allow(controller).to receive(:get_statuses_for_pull_request).and_return("check_status")
controller.instance_variable_set(:@headRefs, {"1234" => "qwerty", "5678" => "asdfg"})
controller.instance_variable_set(:@check_statuses, {})
end

it 'gets and stores the statuses associated with head commits of PRs' do
expect(controller).to receive(:get_statuses_for_pull_request).with("qwerty")
expect(controller).to receive(:get_statuses_for_pull_request).with("asdfg")
controller.retrieve_check_run_statuses
expect(controller.instance_variable_get(:@check_statuses)).to eq({"1234" => "check_status", "5678" => "check_status"})
end
end

describe '#view_github_metrics' do
context 'when user hasn\'t logged in to GitHub' do
before(:each) do
@params = {id: 900}
session["github_access_token"] = nil
end

it 'stores the current participant id and the view action' do
get :view_github_metrics, @params
expect(session["participant_id"]).to eq("900")
expect(session["github_view_type"]).to eq("view_submissions")
end

it 'redirects user to GitHub authorization page' do
get :view_github_metrics, @params
expect(response).to redirect_to(authorize_github_grades_path)
end
end

context 'when user has logged in to GitHub' do
before(:each) do
session["github_access_token"] = "qwerty"
allow(controller).to receive(:get_statuses_for_pull_request).and_return("status")
allow(controller).to receive(:retrieve_github_data)
allow(controller).to receive(:retrieve_check_run_statuses)
end

it 'stores the GitHub access token for later use' do
get :view_github_metrics, {id: '1'}
expect(controller.instance_variable_get(:@token)).to eq("qwerty")
end

it 'calls retrieve_github_data to retrieve data from GitHub' do
expect(controller).to receive(:retrieve_github_data)
get :view_github_metrics, {id: '1'}
end

it 'calls retrieve_check_run_statuses to retrieve check runs data' do
expect(controller).to receive(:retrieve_check_run_statuses)
get :view_github_metrics, {id: '1'}
end
end
end
end

</pre>

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-12-16T03:44:58Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''Teammate Reviews''' under '''View Scores''' functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.
:2. '''Github Metrics''' under '''View Submissions''' page should include a bar chart that shows number of commits by the team throughout the assignment timeline. This will help instructors to get a team overview, and aid grading process.

While this data will not have marks associated directly, it will prove useful to the instructor in differentiating the performance of team members and hence awarding marks as per contribution. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
Previously, view submission does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

==='''Use Case Diagram'''===
[[File:UC_1.png|none|frame|1. Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. 2. Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for viewing GitHub metrics of teams and team members of an assignment.

'''Pre-Conditions:'''

*The Team should have submitted the assignment with a PR link or GitHub repository.

'''Primary Sequence:'''

*The instructor should login.

*The instructor should browse teams for an assignment.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submissions' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metrics table appended at the end, as shown in the figure.

=='''Design Considerations'''==

* The first thing was to determine what metrics we are looking for exactly. These are what the solution supports:
*# Number of commits per user and total per team.
*# Lines of Code added
*# Lines of code deleted.

* The next thing was to narrow down what hosting service for version control we would use. For now, we only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately (changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with (people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
* We also considered if we needed to account for different branches. We only consider the master branch.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories (like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Solution Design'''==

* The Github metrics that need to be integrated with Expertiza were finalized as below. These metrics are captured on a per-user basis:
*# Total number of commits.
*# Lines of Code added
*# Lines of code deleted.
*# Pull Request Status ( includes code climate and Travis CI Build status)
*#User Github metrics:
*## Committer ID
*## Committer Name
*## Committer email ID

* A new link "Github Metrics" is provided under “View Submissions” for an assignment in the instructor view.This link opens a new tab and shows a stacked bar chart for number of commits per user vs submission timeline from assignment creation date to the deadline.
* In "View Scores" for an assignment in the instructor view, under Teammate Reviews tab, a new table for Github Metrics is added, which shows following Github metrics per user:
:Student Name/ID, Email ID, lines of code added, lines of code deleted, number of commits

* For GitHub integration, we have used GitHub GraphQL API v4. We have used github-omniauth gem for authentication/authorization purposes.

* We parse the link to PR to get data associated with it. We have also handled projects which do not have PR link, but just a link to the repository. We excluded expertiza and servo projects as right now a PR link is expected. Future enhancements can look into getting separate GitHub submission links.

* We also show the status of check runs in the View Github metrics view to help instructors view the status of various tools on the repos/PRs without having to go to the actual GitHub page.

=='''Implemented Solution'''==

==='''Files Modified'''===

* app/controllers/auth_controller.rb
* app/controllers/grades_controller.rb
* app/helpers/grades_helper.rb
* app/views/assignments/list_submissions.html.erb
* app/views/grades/_tabbing.html.erb
* app/views/grades/_teammate_reviews_tab.html.erb
* app/views/grades/view.html.erb
* app/views/grades/view_team.html.erb
* config/application.rb
* config/initializers/load_config.rb
* config/initializers/omniauth.rb
* config/routes.rb

==='''Files Added'''===

* app/views/grades/view_github_metrics.html.erb
* config/github_auth.yml

==='''First Change'''===

* A new table "Github Metrics" is added under Manage-> Assignments -> View Scores -> Teammate Reviews. Below is the screenshot of the implementation.

[[File:Ooddafter4.png|none|frame|The GitHub metrics table shows results for each team member]]

==='''Second Change'''===
* The second change is in the View Submissions page, where we have added a link "Github Metrics" to a new page.

[[File:Ooddafter3.png|none|frame|
At present, view submission shows group assignments are submitted as a single submission and 'view github metric' link shows work contribution per teammate.
]]

* The new page appears after clicking on the link "Github metrics", that shows bar chart for # of commits per day. We have also added other relevant information about Pull Request, such as total commits, lines of code added, lines of code modified, PR merge status, check status.

[[File:Capture_3.png|none|frame|The bar chart]]
[[File:Ooddafter10.png|none|frame|The Github summary]]

===='''Code Change in Grades_Controller'''====
* Added below new functions to implement Github Integration in View Submission page
<pre>

def get_statuses_for_pull_request(ref)
url = "https://api.github.com/repos/expertiza/expertiza/commits/" + ref + "/status"
ActiveSupport::JSON.decode(Net::HTTP.get(URI(url)))
end

def retrieve_pull_request_data(pull_links)
pull_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["pull_request_number"] = submission_hyperlink_tokens.pop
submission_hyperlink_tokens.pop
hyperlink_data["repository_name"] = submission_hyperlink_tokens.pop
hyperlink_data["owner_name"] = submission_hyperlink_tokens.pop
github_data = get_pull_request_details(hyperlink_data)
parse_github_data_pull(github_data)
end
end

def retrieve_repository_data(repo_links)
repo_links.each do |hyperlink|
submission_hyperlink_tokens = hyperlink.split('/')
hyperlink_data = {}
hyperlink_data["repository_name"] = submission_hyperlink_tokens[4]
next if hyperlink_data["repository_name"] == "servo" || hyperlink_data["repository_name"] == "expertiza"
hyperlink_data["owner_name"] = submission_hyperlink_tokens[3]
github_data = get_github_data_repo(hyperlink_data)
parse_github_data_repo(github_data)
end
end

def retrieve_github_data
team_links = @team.hyperlinks
pull_links = team_links.select do |link|
link.match(/pull/) && link.match(/github.com/)
end
if !pull_links.empty?
retrieve_pull_request_data(pull_links)
else
repo_links = team_links.select do |link|
link.match(/github.com/)
end
retrieve_repository_data(repo_links)
end
end

def retrieve_check_run_statuses
@head_refs.each do |pull_number, ref|
@check_statuses[pull_number] = get_statuses_for_pull_request(ref)
end
end

def view_github_metrics
if session["github_access_token"].nil?
session["participant_id"] = params[:id]
session["github_view_type"] = "view_submissions"
redirect_to authorize_github_grades_path
return
end

@head_refs = {}
@parsed_data = {}
@authors = {}
@dates = {}
@total_additions = 0
@total_deletions = 0
@total_commits = 0
@total_files_changed = 0
@merge_status = {}
@check_statuses = {}

@token = session["github_access_token"]

@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id

retrieve_github_data
retrieve_check_run_statuses

@authors = @authors.keys
@dates = @dates.keys.sort
end

def authorize_github
redirect_to "https://github.com/login/oauth/authorize?client_id=#{GITHUB_CONFIG['client_key']}"
end

def get_github_data_repo(hyperlink_data)
data = {
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name: \"" + hyperlink_data["repository_name"] + "\") {
ref(qualifiedName: \"master\") {
target {
... on Commit {
id
history(first: 100) {
edges {
node {
id author {
name email date
}
}
}
}
}
}
}
}
}"
}
make_github_api_request(data)
end

def get_pull_request_details(hyperlink_data)
response_data = make_github_api_request(get_query(true, hyperlink_data))

@has_next_page = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]

while @has_next_page
new_response_data = make_github_api_request(get_query(false, hyperlink_data))
response_data["data"]["repository"]["pullRequest"]["commits"]["edges"].push(*new_response_data["data"]["repository"]["pullRequest"]["commits"]["edges"])
@has_next_page = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["hasNextPage"]
@end_cursor = new_response_data["data"]["repository"]["pullRequest"]["commits"]["pageInfo"]["endCursor"]
end

response_data
end

def process_github_authors_and_dates(author_name, commit_date)
@authors[author_name] ||= 1
@dates[commit_date] ||= 1
@parsed_data[author_name] ||= {}
@parsed_data[author_name][commit_date] = if @parsed_data[author_name][commit_date]
@parsed_data[author_name][commit_date] + 1
else
1
end
end

def parse_github_data_pull(github_data)
team_statistics(github_data)
pull_request_object = github_data["data"]["repository"]["pullRequest"]
commit_objects = pull_request_object["commits"]["edges"]
commit_objects.each do |commit_object|
commit = commit_object["node"]["commit"]
author_name = commit["author"]["name"]
commit_date = commit["committedDate"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end

def parse_github_data_repo(github_data)
commit_history = github_data["data"]["repository"]["ref"]["target"]["history"]
commit_objects = commit_history["edges"]
commit_objects.each do |commit_object|
commit_author = commit_object["node"]["author"]
author_name = commit_author["name"]
commit_date = commit_author["date"].to_s
process_github_authors_and_dates(author_name, commit_date[0, 10])
end
organize_commit_dates
end

def make_github_api_request(data)
uri = URI.parse("https://api.github.com/graphql")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
request = Net::HTTP::Post.new(uri.path, 'Authorization' => 'Bearer' + ' ' + session["github_access_token"])
request.body = data.to_json
http.request(request)
response = http.request(request)
ActiveSupport::JSON.decode(response.body.to_s)
end

def organize_commit_dates
@dates.each_key do |date|
@parsed_data.each_value do |commits|
commits[date] ||= 0
end
end
@parsed_data.each {|author, commits| @parsed_data[author] = Hash[commits.sort_by {|date, _commit_count| date }] }
end

def team_statistics(github_data)
@total_additions += github_data["data"]["repository"]["pullRequest"]["additions"]
@total_deletions += github_data["data"]["repository"]["pullRequest"]["deletions"]
@total_files_changed += github_data["data"]["repository"]["pullRequest"]["changedFiles"]
@total_commits += github_data["data"]["repository"]["pullRequest"]["commits"]["totalCount"]
pull_request_number = github_data["data"]["repository"]["pullRequest"]["number"]
@head_refs[pull_request_number] = github_data["data"]["repository"]["pullRequest"]["headRefOid"]

@merge_status[pull_request_number] = if github_data["data"]["repository"]["pullRequest"]["merged"]
"MERGED"
else
github_data["data"]["repository"]["pullRequest"]["mergeable"]
end
end

def get_query(is_initial_page, hyperlink_data)
commit_query_line = if is_initial_page
"commits(first:100){"
else
"commits(first:100, after:" + @end_cursor + "){"
end
{
query: "query {
repository(owner: \"" + hyperlink_data["owner_name"] + "\", name:\"" + hyperlink_data["repository_name"] + "\") {
pullRequest(number: " + hyperlink_data["pull_request_number"] + ") {
number additions deletions changedFiles mergeable merged headRefOid
" + commit_query_line + "
totalCount
pageInfo{
hasNextPage startCursor endCursor
}
edges{
node{
id commit{
author{
name
}
additions deletions changedFiles committedDate
}}}}}}}"
}
end

</pre>

=='''Test Plan'''==

'''Change 1: GitHub metrics in teammate reviews'''

1) Log in as an instructor (instructor6/password)

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate Reviews" tab

6) View data per team member based on different GitHub metrics (e.g. lines of code added/changed/removed etc.)

'''Change 2: Bar chart for # of commits changed by the overall team'''

1) Log in as an instructor (instructor6/password)

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Click on the "Github metrics" link for the team whose metrics you wish to view

5) A new page opens and shows # of commits changed per team member since the start of the assignment, also bottom of the page shows summary from Github submissions.

==='''RSpec Tests'''===

* Following feature tests were added to the Grades_Controller_Spec.rb
<pre>

describe '#get_statuses_for_pull_request' do
before(:each) do
allow(Net::HTTP).to receive(:get) {"{\"team\":\"rails\",\"players\":\"36\"}"}
end

it 'makes a call to the GitHub API to get status of the head commit passed' do
expect(controller.get_statuses_for_pull_request('qwerty123')).to eq({"team" => "rails", "players" => "36"})
end
end

describe '#retrieve_pull_request_data' do
before(:each) do
allow(controller).to receive(:get_pull_request_details).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_pull)
end

it 'gets pull request details for each PR link submitted' do
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1261",
"repository_name" => "expertiza",
"owner_name" => "expertiza"
})
expect(controller).to receive(:get_pull_request_details).with(
{
"pull_request_number" => "1293",
"repository_name" => "mamaMiya",
"owner_name" => "Shantanu"
})
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end

it 'calls parse_github_data_pull on each of the PR details' do
expect(controller).to receive(:parse_github_data_pull).with({"pr" => "details"}).twice
controller.retrieve_pull_request_data(["https://github.com/expertiza/expertiza/pull/1261", "https://github.com/Shantanu/mamaMiya/pull/1293"])
end
end

describe '#retrieve_repository_data' do
before(:each) do
allow(controller).to receive(:get_github_data_repo).and_return({"pr" => "details"})
allow(controller).to receive(:parse_github_data_repo)
end

it 'gets details for each repo link submitted, excluding those for expertiza and servo' do
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "website",
"owner_name" => "Shantanu"
})
expect(controller).to receive(:get_github_data_repo).with(
{
"repository_name" => "OODD",
"owner_name" => "Edward"
})
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD", "https://github.com/expertiza/expertiza", "https://github.com/Shantanu/expertiza]"])
end

it 'calls parse_github_data_repo on each of the PR details' do
expect(controller).to receive(:parse_github_data_repo).with({"pr" => "details"}).twice
controller.retrieve_repository_data(["https://github.com/Shantanu/website", "https://github.com/Edward/OODD"])
end
end

describe '#retrieve_github_data' do
before(:each) do
allow(controller).to receive(:retrieve_pull_request_data)
allow(controller).to receive(:retrieve_repository_data)
end

context 'when pull request links have been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/Shantanu/website/pull/1123"])
controller.instance_variable_set(:@team, teams_mock)
end

it 'retrieves PR data only' do
expect(controller).to receive(:retrieve_pull_request_data).with(["https://github.com/Shantanu/website/pull/1123"])
controller.retrieve_github_data
end
end

context 'when pull request links have not been submitted' do
before(:each) do
teams_mock = double
allow(teams_mock).to receive(:hyperlinks).and_return(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.instance_variable_set(:@team, teams_mock)
end

it 'retrieves repo details ' do
expect(controller).to receive(:retrieve_repository_data).with(["https://github.com/Shantanu/website", "https://github.com/expertiza/expertiza"])
controller.retrieve_github_data
end
end
end

describe '#retrieve_check_run_statuses' do
before(:each) do
allow(controller).to receive(:get_statuses_for_pull_request).and_return("check_status")
controller.instance_variable_set(:@headRefs, {"1234" => "qwerty", "5678" => "asdfg"})
controller.instance_variable_set(:@check_statuses, {})
end

it 'gets and stores the statuses associated with head commits of PRs' do
expect(controller).to receive(:get_statuses_for_pull_request).with("qwerty")
expect(controller).to receive(:get_statuses_for_pull_request).with("asdfg")
controller.retrieve_check_run_statuses
expect(controller.instance_variable_get(:@check_statuses)).to eq({"1234" => "check_status", "5678" => "check_status"})
end
end

describe '#view_github_metrics' do
context 'when user hasn\'t logged in to GitHub' do
before(:each) do
@params = {id: 900}
session["github_access_token"] = nil
end

it 'stores the current participant id and the view action' do
get :view_github_metrics, @params
expect(session["participant_id"]).to eq("900")
expect(session["github_view_type"]).to eq("view_submissions")
end

it 'redirects user to GitHub authorization page' do
get :view_github_metrics, @params
expect(response).to redirect_to(authorize_github_grades_path)
end
end

context 'when user has logged in to GitHub' do
before(:each) do
session["github_access_token"] = "qwerty"
allow(controller).to receive(:get_statuses_for_pull_request).and_return("status")
allow(controller).to receive(:retrieve_github_data)
allow(controller).to receive(:retrieve_check_run_statuses)
end

it 'stores the GitHub access token for later use' do
get :view_github_metrics, {id: '1'}
expect(controller.instance_variable_get(:@token)).to eq("qwerty")
end

it 'calls retrieve_github_data to retrieve data from GitHub' do
expect(controller).to receive(:retrieve_github_data)
get :view_github_metrics, {id: '1'}
end

it 'calls retrieve_check_run_statuses to retrieve check runs data' do
expect(controller).to receive(:retrieve_check_run_statuses)
get :view_github_metrics, {id: '1'}
end
end
end
end

</pre>

=='''Change-log for Reviewers'''==
This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.

* '''Added''' Solution Design for the final implemented design
* '''Added''' Implemented Solution to show feature additions to Expertiza
* '''Added''' Feature test cases for Grades controller

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T02:21:09Z

Sbhoyar:

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T02:19:29Z

Sbhoyar: /* Change-log for reviewers */

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:37:52Z

Sbhoyar: /* Change-log for reviewers */

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:35:40Z

Sbhoyar: /* Problem Statement */

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:35:07Z

Sbhoyar: /* Problem Statement */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

While this data will not have marks associated directly, it will prove useful to the instructor in differentiating the performance of team members and hence awarding marks as per contribution. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in an instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

=='''Change-log for reviewers'''==
This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.

* Updated "Plan of Action" section to include which files and functions will be exactly updated.
* Removed "View submission" changes as per updated requirements(by the Professor).
* Updated "Design Considerations" with results of further investigation and our conclusions as to which approach we will take.

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:26:32Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in an instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

=='''Change-log for reviewers'''==
This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.

* Updated "Plan of Action" section to include which files and functions will be exactly updated.
* Removed "View submission" changes as per updated requirements(by the Professor).
* Updated "Design Considerations" with results of further investigation and our conclusions as to which approach we will take.

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:25:32Z

Sbhoyar: /* Changelog for reviewers */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in an instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

=='''Change-log for reviewers'''==
This section will be removed in the final draft. It is just here for convenience of reviewers to know which sections were majorly updated from last review.

* Updated "Plan of Action" section to include which files and functions will be exactly updated.
* Removed "View submission" changes as per updated requirements(by the Professor).
* Updated "Design Considerations" with results of further investigation and our conclusions as to which approach we will take.

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:22:05Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in an instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

=='''Changelog for reviewers'''==
* Updated "Plan of Action" section to include which files and functions will be exactly updated.
* Removed "View submission" changes as per updated requirements(by the Professor).
* Updated "Design Considerations" with results of further investigation and our conclusions as to which approach we will take.

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:05:47Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in an instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:05:14Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades controller(grades_controller.rb)'''. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the same controller. It will return the GitHub data which will be stored in a instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-21T00:03:57Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem '''Octokit''' to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use '''omniauth-github''' gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the '''Grades''' controller. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the '''view''' action in the controller. It will return the GitHub data which will be stored in a instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the '''_teammate_reviews_tab.html.erb''' partial under '''grades''' view-folder. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the '''bar-chart''' function in the '''grades''' controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-20T23:59:19Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing was to determine what metrics we are looking for exactly. These are what the requirements document suggested:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing was to narrow down what hosting service for version control we would use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also considered whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all). We have decided to go with parsing the links as giving the link to PR is anyway mandatory.

* An important question was whether we needed to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this would likely cause issues with stale information and would have been difficult to maintain.
*# Having a db was redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach was to take snapshots of metrics and store them in the db right on the submission deadline of projects. This would allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects would put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might have been good for grading purposes but wouldn't have helped with determining the current status of a project.
*# We have decided against using our own tables for this data and will be getting the GitHub data on-demand directly using the GitHub API.
*# All that said, we might will maintain some meta-data in our db around what metrics we are retrieving(like LOCs, commiters, frequencies etc.)
* We also considered if we needed to account for different branches. The plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show. We will be keeping this in mind when we implement the views.

* With respect to showing GitHub metrics in the View scores page, it would have been very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, we will be showing a new table below it to display the metrics. This will allow the instructor full view of how teammate rated each other and how that maps to factual information from GitHub.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem Octokit to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use omniauth-github gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the Grades controller. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the "view" action in the controller. It will return the GitHub data which will be stored in a instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the "_teammate_reviews_tab.html.erb" partial under "grades" view. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the "bar-chart" function in the "grades" controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-20T23:51:23Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# We have decided against using our own tables for this data.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* In line with our investigation, we will be using the official gem Octokit to get GitHub data from API. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we will alternate a view between the latest data and data till submission deadlines.
* Unfortunately, GitHub only allows 60 public GET requests per hour and 5000 if authenticated. We thus intend to use omniauth-github gem to integrate GitHub OAuth to authenticate the instructor.
* There is [https://developer.github.com/v3/ ample] documentation for GitHub API. The GitHub API returns JSON and we have been able to retrieve sample data using it.
* The "Teammate Review" section will get an additional table, in the View Scores page. We will be introducing our logic in the Grades controller. A new function will be created that will use the Octokit functions to retrieve the required data for the entire team.
* Our new function will be called from the "view" action in the controller. It will return the GitHub data which will be stored in a instance variable accessible to the views.
* We then implement the view. To show the GitHub metrics, we will be modifying the "_teammate_reviews_tab.html.erb" partial under "grades" view. The instance variable will be accessible here and will be used to pass data to the table.
* For charting, we will use the "bar-chart" function in the "grades" controller, also used in a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].
* The table will be shown below the actual "Teammate Review" table as per the requirements.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-20T23:22:13Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# We have decided against using our own tables for this data.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-17T01:14:10Z

Sbhoyar: /* Design Principles */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing our end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-16T02:56:47Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-15T03:00:05Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-15T02:59:36Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-15T02:53:25Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like GitHub.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from GitHub.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like GitHub. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like GitHub in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on GitHub is a solution, but that is inefficient from instructor's/reviewer's perspective as there are many assignments, submissions, and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to the complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “GitHub Metrics” under “View Submissions” in the instructor view.
:A line chart of the number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support GitHub integration due to its popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use GitHub.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for GitHub repository link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with [https://docs.google.com/document/d/1gqKM6pOo02MnRIDRGOxgdKy0EmyWMp0uiK_czQzTsF8/edit this] schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with GitHub and then show the results. So we end up hitting GitHub API anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on the server. Also, for open source projects, this would mean that we don’t have the latest data to work with(people will keep committing past the deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing GitHub metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public GitHub profiles as students may use a different name. So instead of appending GitHub data to Teammate reviews table, it would be easier to just add a new tab called "GitHub Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the project's repositories(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get GitHub data and if we are allowed to use them. The idea is to always get the latest data from GitHub and show it to the user. For grading purposes, we could alternate a view between the latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from GitHub. There is [https://developer.github.com/v3/ ample] documentation for this. The GitHub API returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “GitHub Metrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock-up view to display this data in a table.
* We then implement the view. For charting, a suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'GitHub contribution metric' in teammate review. ]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'GitHub contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing, and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able to see the team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in the figure.
*Instructor will be able to see the work done by each student in 'Teammate Review Tab' with new metric appended at the end, as shown in the figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling GitHub data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the GitHub metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|The GitHub metrics are colored in blue]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|The GitHub metrics are colored in blue]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team.

[[File:Ssdtfinalss11.png|none|frame|The GitHub metrics column shows the line graphs for every team]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Test Plan'''==

'''Subtask 1: GitHub metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "GitHub metrics" tab

6) View data based on different GitHub metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different GitHub metrics (e.g. lines of code added/changed/removed etc.) portrayed in a different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by the overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added GitHub metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ GitHub API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:30:44Z

Sbhoyar: /* Current Scenario */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like Github. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like Github in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows peer review amongst teammates. Currently, however, there is no way to validate and verify these reviews.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:23:48Z

Sbhoyar: /* Database Design */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give a more realistic view of the work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like Github. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like Github in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like the number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications.
As number of commits and LOCs(lines of code) will keep of changing and stale data does not seem to be of significant benefit for analysis, we will be concentrating on latest state of project repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:19:47Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like Github. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like Github in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page.

====='''Approach 1'''=====

* We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:18:48Z

Sbhoyar: /* Problem Statement */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submissions''' functionality provides a list of submissions of all the teams for a particular assignment. We can add a new column to show work distribution within the whole team using external tools like Github. A new column will display this information using line-graphs and other statistics.

:2. '''Teammate Reviews''' functionality in the View Scores page gauges teammate views on how much other team members contributed to the project. We need to augment this data with data from external tools like Github in order to validate that feedback. New metrics will be appended under each student data under the same functionality.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

====='''Approach 1'''=====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:12:29Z

Sbhoyar: /* Detail Design */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Design Detail'''===

===='''First Change'''====

====='''Approach 1'''=====

* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

====='''Approach 2'''=====

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

===='''Second Change'''====
* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T03:08:30Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:51:58Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use Case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

====='''Use Case Diagram Details'''=====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

==='''Database Design'''===

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:48:55Z

Sbhoyar: /* Use case Diagram */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

=='''Database Design==

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:38:19Z

Sbhoyar: /* Current Scenario */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate.
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission ==
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

=='''Database Design==

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:37:51Z

Sbhoyar: /* Current Scenario */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up.

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission ==
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

=='''Database Design==

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:37:37Z

Sbhoyar: /* Current Scenario */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done by the team as a whole. This does not show work contribution per teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. Currently, however, there is no way to back those review up. ==

]]

Checking commits performed by each team member on Github is a solution, but that is inefficient from instructor/reviews perspective considering the large number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission ==
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

=='''Database Design==

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

CSC/ECE 517 Fall 2018- Project E1858. Github metrics integration

2018-11-14T02:32:14Z

Sbhoyar: /* Problem Statement */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.

This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Proposed Solution Design'''==
==='''Design Considerations'''===

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

==='''Plan of action'''===

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

===='''Use case Diagram'''====
[[File:Ssdtusecasemerge.png|none|frame|Use Case diagram of two approaches to append 'Github contribution metric' in teammate review. ==]]
[[File:Ssdtusecase323.png|none|frame|Use Case diagram explaining approach to add new column 'Github contribution metric' in 'View submission ==
]]

===='''Use Case Diagram Details'''====
'''Actors:'''

*Instructor: This actor is responsible for creating assignments and adding students to the assignment.

*Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

'''Database:'''

*The database where all the data of Expertiza is getting stored.

'''Pre Conditions:'''

*The Student should submit the assignment and self-review.

*The other students should submit the reviews of the work submitted.

'''Primary Sequence:'''

*The student should login.

*The student should browse and upload the assignment.

*The student should submit the assignment.

*The student should submit teammate-reviews.

'''Post Conditions:'''

*Instructor will be able see team contribution done by each team member in 'View Submission' page using graph diagrams, as shown in figure.
*Instructor will be able see work done by each student in 'Teammate Review Tab' with new matric appended at the end, as shown in figure.

==='''Design Principles'''===

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

==='''Detail Design'''===
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:Ssdtviewscore.png|none|frame|caption]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:
[[File:Ssdtviewsub.png|none|frame|caption]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

[[File:Ssdtfinalss11.png|none|frame|caption]]

=='''Database Design==

As of now, we do not have plans for database modifications, as we doing our research around exporting data using API's and displaying.
As number of commits and number of line changing will keep of changing and stale data does not seem of significant benefit for future analysis, we are concentrating more on fetching current data repositories.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

E1858 Github metrics integration

2018-11-13T06:42:36Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Current Scenario'''==

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase12.png|none|caption
File:Ssdtusecase22.png|none|caption
File:Ssdtusecase32.png|none|caption
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

E1858 Github metrics integration

2018-11-13T06:41:52Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''View Submissions''' and '''Teammate Reviews''' (under View Scores) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase12.png|none|caption
File:Ssdtusecase22.png|none|caption
File:Ssdtusecase32.png|none|caption
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

E1858 Github metrics integration

2018-11-13T06:28:42Z

Sbhoyar: /* Plan of action */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase12.png|none|caption
File:Ssdtusecase22.png|none|caption
File:Ssdtusecase32.png|none|caption
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

E1858 Github metrics integration

2018-11-13T06:28:29Z

Sbhoyar:

E1858 Github metrics integration

2018-11-13T06:25:35Z

Sbhoyar: /* References */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==
<gallery>
File:Ssdtusecase12.png|none|caption
File:Ssdtusecase22.png|none|caption
File:Ssdtusecase32.png|none|caption
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan'''==

'''Subtask 1: Github metrics in teammate reviews'''

'''Test plan for proposed solution 1:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Github metrics" tab

6) View data based on different Github metrics (e.g. lines of code added/changed/removed etc.) for each teammate

'''Test plan for proposed solution 2:'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View scores" icon for the assignment of your choice

4) Select the team for which you wish to view scores

5) Go to "Teammate reviews" tab

6) Select the student for whom you wish to view teammate review

7) Below the usual criteria, view criteria for different Github metrics (e.g. lines of code added/changed/removed etc.) portrayed in different color scheme (light blue)

'''Subtask 2: Line chart for # of lines changed by overall team'''

1) Log in as an instructor

2) Navigate to assignments through Manage --> Assignments

3) Select "View submissions" icon for the assignment of your choice

4) Select the team whose submissions you wish to view

5) A newly added Github metrics column is added to show # of lines changed since the start of the assignment

=='''References'''==

[http://wiki.expertiza.ncsu.edu/index.php/Main_Page Expertiza_wiki]

[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader E1815:_Improvements_to_review_grader]

[https://github.com/expertiza/expertiza/pull/1179 Expertiza_PR_1179]

[https://www.youtube.com/watch?v=HHdta64VHcY Expertiza_PR_1179_Video]

[https://developer.github.com/v3/ Github API documentation]

E1858 Github metrics integration

2018-11-13T06:23:58Z

Sbhoyar: /* References */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==
<gallery>
File:Ssdtusecase12.png|none|caption
File:Ssdtusecase22.png|none|caption
File:Ssdtusecase32.png|none|caption
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References'''==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader

https://github.com/expertiza/expertiza/pull/1179

https://www.youtube.com/watch?v=HHdta64VHcY

https://developer.github.com/v3/

E1858 Github metrics integration

2018-11-13T06:20:05Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

[[File:Ssdtusecase12.png|none|caption]]
[[File:Ssdtusecase22.png|none|caption]]
[[File:Ssdtusecase32.png|none|caption]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T06:17:22Z

Sbhoyar: /* Design Pattern */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase1.png|none|frame|image3
File:Ssdtusecase2.png|none|frame|image2
File:Ssdtusecase3.png|none|frame|image1
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Principles'''==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively). We intend to follow the same when implementing out end-point for pulling Github data.

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T06:14:54Z

Sbhoyar: /* Design Pattern */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase1.png|none|frame|image3
File:Ssdtusecase2.png|none|frame|image2
File:Ssdtusecase3.png|none|frame|image1
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T06:14:39Z

Sbhoyar: /* Use case Diagram */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Use case Diagram'''==

<gallery>
File:Ssdtusecase1.png|none|frame|image3
File:Ssdtusecase2.png|none|frame|image2
File:Ssdtusecase3.png|none|frame|image1
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T06:14:15Z

Sbhoyar:

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

==Use case Diagram==

<gallery>
File:Ssdtusecase1.png|none|frame|image3
File:Ssdtusecase2.png|none|frame|image2
File:Ssdtusecase3.png|none|frame|image1
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:54:16Z

Sbhoyar: /* Test Cases */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

<gallery>
File:Ssdt_usecase1.png|Item 1
File:Ssdt_usecase2.png|a link to [[Help:Contents]]
File:Ssdt_usecase3.png
</gallery>

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Plan==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:51:19Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

[[File:Ssdt_22.png|none|frame|
== At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate. ==
]]

[[File:Ssdt_123.png|none|frame|
== Teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews. ==

]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* With respect to showing Github metrics in the View scores page, it will be very difficult to map Expertiza users and their names to public Github profiles as students may use different name. So instead of appending Github data to Teammate reivews table, it would be easier to just add a new tab called "Github Metrics" and show the results there.

* The instructors will need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

[[File:E1799_design_2.jpg]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Cases==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:43:15Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

*At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate.

[[File:Ssdt_12.png|border]]

*Also teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews.

[[File:Ssdt_22.png|border]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

:We are assuming these metrics are needed on a per-user basis.

:The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
:A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* The instructors need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

[[File:E1799_design_2.jpg]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Cases==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:40:38Z

Sbhoyar: /* Design Considerations */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

*At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate.

[[File:Ssdt_12.png|border]]

*Also teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews.

[[File:Ssdt_22.png|border]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

We are assuming these metrics are needed on a per-user basis.

The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* The instructors need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

[[File:E1799_design_2.jpg]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Cases==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:37:18Z

Sbhoyar: /* Database Design */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate.

[[File:Ssdt_12.png]]

Also teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews.

[[File:Ssdt_22.png]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

We are assuming these metrics are needed on a per-user basis.

The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* The instructors need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

[[File:E1799_design_2.jpg]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Cases==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page

E1858 Github metrics integration

2018-11-13T05:37:06Z

Sbhoyar: /* Database Design */

=='''Introduction'''==

===Problem Statement===

Expertiza provides '''view submission''' and '''teammate review''' (under view score) functionality for each assignment.
Purpose of this project is to augment existing assignment submissions with data that can give more realistic view on work contribution of every team member using external tools like Github.
This external data may include: number of commits, number of lines of code modified, number of lines added, number of lines deleted from each group’s submitted repository link from Github.

:1. '''View submission''' provides list of submission of all the teams for that particular assignment. We can add new column to show work distribution among the whole team from external tools like Github. This new column will provide work distribution on whole team using bar graphs and other statistics.

:2. '''Teammate Reviews''' functionality gauges team mates views on how much each team member contributed, but we would like to augment this data with data from external tools like Github. New metrics will be appended under each student data.
for example,
This information should prove useful for differentiating the performance of team members for grading purposes. Overall data for the team, like number of committers and number of commits may also help instructors to predict which projects are likely to be merged.

=='''Overall Description==

===Current Scenario===

At present, group assignments are submitted as a single submission that shows work done as a whole team. This does not show work contribution done by each teammate.

[[File:Ssdt_12.png]]

Also teammate review shows views of each teammate on every other teammates work. That might leads to discrepancy while validating the reviews.

[[File:Ssdt_22.png]]

Checking commits performed by each team member on Github is one of the solution, but that does not seems efficient from instructor/reviews perspective considering huge number of assignments, number of submissions and tight deadlines.

=='''Design Considerations'''==

* The first thing is to determine what metrics we are looking for exactly. These are what the requirements document suggests:
*# Total number of commits.
*# Number of files changed.
*# Lines of Code added
*# Lines of code modified.
*# Lines of code deleted.
*# LOCs that survived until final submission - (exclude from MVP due to complexity and lower priority).

We are assuming these metrics are needed on a per-user basis.

The requirement document expects a view to be created for viewing the metrics under a new tab “Github Metrics” under “View Submissions” in the instructor view.
A line chart of number of lines vs time needs to be included in the view.

* The next thing is to narrow down what hosting service for version control we will use. For now, the plan is to only support Github integration due to it popularity, ease-of-use and API documentation. Future projects could add in support for Gitlab and others, though it is far easier to just require that all students use Github.
*# The main impact of this change will be that all submission repositories need to be made public as we need access to pull in the data.
*# We also need to consider whether to ask students for github repo link separately(changes to views) or to parse all the uploaded links and determine the correct one (extra logic, students uploading multiple links or not giving links at all).

* An important question is whether we need to store metric information in our own db at all.
*# An older investigation came up with this schema, but this is likely to cause issues with stale information and will be difficult to maintain.
*# Having a db is redundant as every time a user wants to see the metrics, we would need to sync the db with Github and then show the results. So we end up hitting Github api anyway.
*# An alternative to the above approach is to take snapshots of metrics and store them in the db right on the submission deadline of projects. This will allow for fairer grading by making sure we pull in data at the correct moment. Unfortunately, doing this for so many projects could put a lot of load on server. Also, for open source projects this would mean that we don’t have the latest data to work with(people will keep committing past deadline). Thus, this approach might be good for grading purposes but doesn’t help with determining the current status of a project.
*# All that said, we might need to maintain some meta-data in our db- investigation pending.

* We also need to consider if we need to account for different branches. The initial plan is to only consider the master branch.

* A suggestion was also to make sure that there isn’t a lot of padding in the tables we show.

* The instructors need to spell out exact guidelines for committing to the projects repos(like everyone should commit their own code, keep the master as PR branch, commit regularly, be mindful of squashing too many commits for one user), so that we can have proper and correct data and, also so that students can’t weasel their way out later claiming they worked but forgot or didn’t know.

=='''Plan of action'''==

* The first thing is to investigate if there are any rails gems(Octokit/Unicorn maybe?) to get Github data and if we are allowed to use them. The idea is to always get the latest data from Github and show it to the user. For grading purposes, we could alternate a view between latest data and data till submission deadlines.
* If there are no such gems or if we aren’t allowed to use gems, we need to write to mock endpoints to understand out how to get data from github. There is [https://developer.github.com/v3/ ample] documentation for this. The github api returns JSON which can be easily parsed.
* We then need to figure out how to load this data(AJAX? or gem?) when instructor selects “Github Rubrics”(if we do this) or "Teammate Review" in the View Scores page. Similar data needs to be loaded in the View Submissions page.
* We will need a mock up view to display this data in a table.
* We then implement the view. For charting, suggestion is to use a previous project for charting- [https://www.youtube.com/watch?v=HHdta64VHcY 1] [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2018_E1815:_Improvements_to_review_grader 2] [https://github.com/expertiza/expertiza/pull/1179 3].

=='''Application Flow'''==

[[File:E1799_design_1.jpg]]

==Use case Diagram==

[[File:E1799_design_2.jpg]]

Actors:

:Instructor: This actor is responsible for creating assignments and adding students to the assignment.

:Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

Database:

:The database where all the data of Expertiza is getting stored.

All the other use cases are implemented except “View Scores with self-review column and composite score”

Use Case: View score with self-review column and composite score

Pre Conditions:

1.The Student should submit the assignment and self-review.

2.The other students should submit the reviews of the work submitted.

Primary Sequence:

1.The student should login.

2.The student should browse and upload the assignment.

3.The student should submit the assignment.

4.The student should submit the self-review.

5.The student should choose your scores to view the score.

6.The student should be able to see the peer-review score with the self-review scores and composite score.

Post Conditions:

1.The self-review and composite score will be visible on the page.

=='''Database Design==

1.As of now, we do not have plans for database modifications.

=='''Design Pattern==

* MVC – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view and controller, respectively).

* Dry Principle – We are trying to reuse the existing functionalities in Expertiza,thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

* Strategy design pattern –

=='''Proposed solutions'''==
* The first change would would be under the "Teammate Review" tab in the "View Scores" page. We either add new rows to the table, one for each of the Github metrics, and display the results for every student in the project as mocked up here:

[[File:x.png]]

Or we might add a new tab called "Github Metrics" in the View scores page and then show the metrics and the results in tabular format as mocked up:

[[File:x.png]]

* The second change is in the View Submissions page, where we intend to add a new column to the table that shows a chart per assignment team. This chart will show proportion of contributions per team member.

=='''Implementation==

The last column has the self review scores. The composite score is calculated and is displayed at the bottom of the table.

[[File:proper view.png]]

The composite score is displayed beside the average peer review score.

[[File:alternate_view.png]]

=='''Test Cases==

{|border=1
|Test case ID
|Test Objective
|PreConditions
|Steps
|Test Data
|Expected Result
|Post Conditions
|-

|E1799_Test_Case_1
|Successfully displaying self review scores in the heat map view
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The self review column is displayed together with the other peer reviews in the end.
|The student is able to see the self review column.
|-

|E1799_Test_Case_2
|Successfully calculating the proper composite score.
|1. The student should do the self review.
2. The score of the self review should be present in the database.

3. The function is implemented to calculate the composite score using average of the peer reviews and the self review.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is returned by the method.
|The method is able to return the expected composite score.
|-

|E1799_Test_Case_3
|Successfully displaying of composite score in the heat map view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to view scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the heat map view.
|The student is able to see the composite score.
|-

|E1799_Test_Case_4
|Successfully displaying of composite score in alternate view.
|1. The student should do the self review.
2. The scores of the self review should be present in the database.
|1. Login to the Expertiza Application.
2. Submit the assignment created by the instructor by browsing it.

3. Choose the self review option and complete the self review.

4. Go to alternate scores option.
|Submit the relevant assignment before doing the self review.
|The composite score is displayed in the alternate scores view.
|The student is able to see the composite score.
|-

|}

=='''References==

http://wiki.expertiza.ncsu.edu/index.php/Main_Page