E1929 Visualizations for Instructors: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
 
(135 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==
Expertiza is an online assignment grading platform. Instructors can create assignments and implement peer reviews for submitted assignments. This project concerns the creation of a system for visualizing student performance on those assignments, primarily as graded in peer reviews. Graphs will be made to show various rubric criteria and the class' performance on the criteria. If the criteria are the same for multiple stages of review, an instructor should be able to compare performance over time or between reviews. Our task is to provide an interactive visualization or a table for instructors that shows how their class performed on selected rubric criteria. Such feature would be immensely helpful for instructors as it would assist them to identify what they need to focus more attention on. For example, creating a graph showing the average scores for all or a certain subset of main rubric criteria (questionnaire). If the average score of the class on selected criteria (question) is low means the instructor can emphasize more on the learning materials related to it.
This [http://wiki.expertiza.ncsu.edu/index.php/Expertiza_documentation Expertiza] project, completed in the Spring of 2019, aimed to improve the charts instructors could view to see the grade statistics for a given assignment. Two charts already existed, one showing the grade distribution for all teams, and another simply showing the class average grade. Our goal for this project was to add a third chart, using [https://developers.google.com/chart/ Google Charts], which would show grade statistics for the various [https://en.wikipedia.org/wiki/Rubric_(academic) rubric] grades within that assignment. The chart is interactive so the user (instructor) can toggle which rubric criteria, and which statistics, to display.
 
 
Along with displaying the rubric criteria for a single assignment, we aimed to add a feature which would allow the comparison of compatible rubric grades from two different assignments. Again, this functionality is interactive, allowing the instructor to choose which statistics will be populated in the chart.
 
 
By viewing our new visualization of rubric grades, an instructor can better judge which aspects of the course are well-understood, and which may need a bit more attention.




Line 6: Line 12:




== Proposed Changes ==
== Problem Statement ==
The visualizations will be implemented as either a single or stacked bar chart with a bar for each of the selected criteria to be observed. If a single bar, then the height of the bar will be the total class average, but a stacked bar chart may be better to show the percentage of the class that received each score.
'''E1929 - Visualizations for Instructors''' - Class performance on specific rubrics
 
An interactive visualization or table that shows how a class performed on selected rubric criteria would be immensely helpful. It would show the instructor what he / she will need to focus more attention on. For example, could you create a graph showing the range and clustering of scores for the 5 main rubric criteria? And, if these same 5 criteria are used in the preliminary and final assignments, it would be nice to be able to compare performance between assignments in a visualization that showed the class results on 3 separate artifacts.
 
Webpage: login as instructor -> Manage -> Assignments -> View scores
 


=== Old Charts ===
[[File:Existing_Assignment_Grades_Wed.png|frame|centre|'''Figure 1: Existing assignment grade charts - '''The existing assignment grade charts show the average grade for the assignment and the grade distribution. During this project, we aimed to add a third chart which shows the distribution for specific rubric criteria within the assignment.]]


1. On clicking Manage and then on assignments, following page appears.


2. Click on 'view score' icon of an assignment. The summary report page of the selected assignment comes up.
== Accomplished ==


3. Following are mockup screens which we wish to create:
=== Executed on Plans ===
We successfully executed all of the ideas we had during the planning phase:
* Add new feature to show mean and median data for rubric criteria in a given assignment
* Add new feature to show comparison
* Performed UI testing to ensure features were operational, including edge cases
* Added and ran RSpec and Capybara automated tests


a) Instructor would select the round and rubric criteria of the assignment for which he/she wants to view the class performance.
b) The bar graph of the class performance for those criteria would be displayed.


== Project Design ==
=== New Charts Inserted ===
The changes made to the expertiza project will primarily include HTML/ERB changes to the view files to accommodate the added charts on the page and the necessary javascript to allow responsive design. Brief controller modifications will be made to facilitate database filtering to get the displayed data.
[[File:AnalyzeAssignmentWed.png|frame|centre|'''Figure 2: New Rubric Statistic Visualization - '''Added in the center of the existing two charts is an interactive chart the instructor can use to see the mean or median scores for whichever criteria they select. This is done on the "Analyze Assignment" tab, which is selected by default when the "View Scores" page is loaded.]]


=== Design Flow ===
The flowchart representing graphical flow of an instructor visiting view scores under assignments is given below:


[[File:A3.png]]
[[File:CompareAssignmentsWed.png|frame|centre|'''Figure 3: New Rubric Cross-Assignment Comparison Visualization - '''The instructor can see a comparison between selected criteria between the current assignment and the selected assignment. The rubric criteria must be compatible in order for it to even show up in the selection list. This comparison visualization is done on the "Compare Assignments" tab which must be selected after the "View Scores" page has been loaded.]]


=== Tools and Design Choice ===
We have used the lightweight [https://developers.google.com/chart/ Google Charts] library for displaying the chart data on the page, with standard HTML for all of the options and dropdowns for option selection. Google Charts was chosen because of its high compatibility, full option set, and comparable graphical quality to the rest of expertiza while keeping a small JS footprint, which should help prevent slow page responsiveness.


=== Visualization ===
=== Extra Issues Fixed ===
The following graph shows the view of the 'view scores' page after the modifications. The instructor selects a subset of rubric criteria for which he/she wants to know how a class performed for a particular round. A bar graph of the average score of the class for that subset of criteria is displayed.


The above graph shows an average score of the class for 10 rubric criteria in Round 1 for assignment "OSS project/Writing assignment 2" selected by the instructor. A live demo with randomly generated data can be found on [https://jsfiddle.net/Jereman/5uxqr92y/ JSFiddle]
* Refactored [https://github.com/gshugh/expertiza/blob/OSS-HW4/app/views/grades/_team_charts.html.erb _team_charts.html.erb] partial view to use partials for each graphic. Previously both charts were defined together in _team_charts.html.erb, but now each graphic has its own partial view ([https://github.com/gshugh/expertiza/blob/OSS-HW4/app/views/grades/_team_charts_averages.html.erb _team_charts_averages.html.erb], [https://github.com/gshugh/expertiza/blob/OSS-HW4/app/views/grades/_team_charts_distribution.html.erb _team_charts_distribution.html.erb], [https://github.com/gshugh/expertiza/blob/OSS-HW4/app/views/grades/_team_charts_rubric_stats.html.erb _team_charts_rubric_stats.html.erb]). This allows more flexibility going forward, if it's decided one chart should move to a new place on the page, or to a new page.


== Test Plan ==
The team plans to perform both automated tests, using the RSpec framework, and manual tests of the user interface (UI), using the app.


=== RSpec Framework Tests ===
=== Code Changes ===
The RSpec Testing Framework, automated testing, will be used to verify the Models of the Expertiza Rails Web application feature set. Since this feature is dealing with Visualizations (charts) that is intimately tied with an Active Record Models, we will seed the Testing Database with known data via RSpec. These changes will be automatically rolled-back once the testing is complete. 
[https://github.com/expertiza/expertiza/pull/1440/files Files Added/Modified]
 
Added: 10 + Modified: 11 = Total: 21


=== Capybara Tests ===
* app/assets/javascripts/rubric_stats.js '''(new)'''
Another automated testing framework that will be used is Capybara. Capybara is an browser type test to simulate a user clicking through your site. We will use this testing framework to verify that our charting object is present on the page and contains the seeded data that we had loaded.
* app/controllers/grades_controller.rb
* app/helpers/assignment_stats_helper.rb '''(new)'''
* app/models/assignment.rb
* app/models/assignment_questionnaire.rb
* app/models/assignment_stats.rb '''(new)'''
* app/models/criterion_stats.rb '''(new)'''
* app/models/question.rb
* app/models/questionnaire.rb
* app/models/review_round_stats.rb '''(new)'''
* app/models/score_view.rb
* app/views/grades/_team_charts.html.erb
* app/views/grades/_team_charts_averages.html.erb '''(new)'''
* app/views/grades/_team_charts_distribution.html.erb '''(new)'''
* app/views/grades/_team_charts_rubric_stats.html.erb '''(new)'''
* spec/controllers/grades_controller_spec.rb
* spec/factories/factories.rb
* spec/features/grade_interface_spec.rb '''(new)'''
* spec/features/helpers/grade_interface_helper.rb '''(new)'''
* spec/models/assignment_spec.rb
* spec/models/questionnaire_spec.rb


=== UI Tests ===
In addition to the automated tests above we will also perform manual testing of the newly added features to include:


# Chart is displaying correctly
== Testing ==
## Bars are showing up where expected
We performed automated tests using RSpec Framework and Capybara. In addition, we performed manual tests of the user interface (UI), by using the app.
## Bar annotations are showing the expected value
## Criteria labels are for the correct bar and displaying correct values
## Hover text is displaying the correct values
## Null values are not present on the chart
## Correct colors are used for the multi-round view
# Show Labels checkbox works as expected
# Round Criteria is displaying correctly
## Round dropdown menu shows all rounds for the assignment
## Selecting a round changes the criteria checkboxes
## All checkboxes are displayed with appropriate text
## Checkboxes correctly remove or add criterion bars to the chart


== Files Involved ==
=== RSpec Framework Tests ===
We have implemented a new partial file criteria_charts to the team_chart that display the bar graph with existing data collected by the grades controller methods to the view page.
The RSpec Testing Framework, automated testing, was used to verify the models of the Expertiza web application feature set. Since this feature is dealing with visualizations (charts) that are intimately tied with Active Record models, we seeded the testing database with known data via FactoryBot gem. These changes were automatically rolled-back once the testing was complete.


Modified files:
RSpec was used to increase/maintain automated test coverage for the methods that we added into assignment and questionnaire models. Mocks and stubs were utilized in order to decouple the model under test from dependencies of other models. This will allow us to identify easily if our model updates are causing an issue.
<div style="display: inline-block">
<pre>
app/controllers/grades_controller.rb


app/views/grades/_teams.html.erb
=== Capybara Tests ===
RSpec Capybara was used to provide integration testing of the charts. The Test Database was seeded with the appropriate information for executing the results of the charts. An additional GEM was needed, webdrivers, to utilize the predefined drivers of Selenium. The default web driver, Rack-Test, does not execute java scripts. Since the graphs that we are using require java script, thus soliciting the necessity of Selenium with its predefined web driver, selenium_chrome_headless. This provides the required functionality to process the java scripts and to automatically ensure that they exist in the DOM.  


app/views/grades/_criteria_charts.html.erb
In order to use the headless web driver with the seeded Test Database, a different database strategy was required. As a result, for this test only, the database strategy was changed to truncation. Ruby 2.2.7 is used in the current version of Expertiza. This version does not allow for sharing of the database thread, and thus requires the to be written to the database outside of a transaction block. This allows for the headless web driver to see that the data is persisted the Test Database and is able to run as normal. Once the test is completed, the database tables are truncated and all data is removed.


app/views/grades/_team_charts.html.erb
=== UI Tests ===
</pre>
In addition to the automated tests above we also performed manual testing of the newly added features to include:
</div>


=== app/views/grades/_criteria_charts.html.erb ===
* The new feature properly initializes
<html>
*# Analyze Assignment tab is loaded by default
  <head>
*# Chart is in between the other two existing charts
<%= content_tag :div, class: "chartdata_information", data: {chartdata: @chartdata} do %>
*# Round 1, all Criteria selected, and Mean are selected by default
    <% end %>
* Analyze Assignment tab operating correctly
<%= content_tag :div, class: "text_information", data: {text: @text} do %>
*# Bars are showing up where expected
    <% end %>
*# Bar annotations are showing the expected value
    <%= content_tag :div, class: "minmax_information", data: {minmax: @minmax} do %>
*# Criteria labels are for the correct bar and displaying correct values
    <% end %>
*# Hover text is displaying the correct values
<script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
*# Null values are not present on the chart
    <script type="text/javascript">
*# Round dropdown menu shows all rounds for the assignment
      google.charts.load('current', {packages: ['corechart', 'bar']});
*# Selecting a round changes the criteria checkboxes
      //Display Options
*# All checkboxes are displayed with appropriate text
      var showLabels = true;
*# Checkboxes correctly remove or add criterion bars to the chart
      var barColors = [ // Other colors generated from the expertiza base red
* Compare Assignments tab operating correctly
          '#A90201',    // using paletton.com
*# Compare Assignments tab is only displayed if compatible assignments exist
          '#018701',
*# Clicking Compare Assignments tab loads that tab
          '#016565',
*# (All tests from Analyze Assignment tab apply to this tab as well)
          '#A94D01'
*# The two colors for assignment comparison are correct
      ];
      function getData(){ //Loads all chart data from the page
        chartData = $('.chartdata_information').data('chartdata');
        chartText = $('.text_information').data('text');
        chartRange = $('.minmax_information').data('minmax');
for (var i = 0; i < chartData.length; i++){ //Set all the criteriaSelected to true
          var criteria = [];
          for (var j = 0; j < chartData[i].length; j++){
            criteria.push(true);
          }
          criteriaSelected.push(criteria);
        }
    }
      function generateData() { //Generates random data for testing
        var rounds = 3;
        for(var i = 0; i < rounds; i++) {
  var criteriaNum = Math.floor(Math.random() * 5 + 5);  //Random number of criteria
          var round = [];
          var criteria = [];
          for(var j = 0; j < criteriaNum; j++) {
  round.push(Math.floor(Math.random() * 101));  //Random score for each criterion
            criteria.push(true);  //Everything starts out true
          }
          chartData.push(round);
          criteriaSelected.push(criteria);
        chartOptions = { //Render options for the chart
          title: 'Class Average on Criteria',
          titleTextStyle: {
  fontName: 'arial',
            fontSize: 18,
            italic: false,
            bold: true
        chart = new google.visualization.ColumnChart(document.getElementById('chart_div'));
        var checkBox = document.getElementById("labelCheck");
        checkBox.checked = true;
        checkBox.style.display = "inline";
        updateChart(currentRound);
        loadRounds(); }
function updateChart(roundNum) { //Updates the chart with a new round number and renders
        currentRound = roundNum;
        renderChart();
        loadCriteria();
      function renderChart() { //Renders the chart if changes have been made
          var data = loadData();
chartOptions.vAxis.viewWindow.max = 5;
          chartOptions.vAxis.viewWindow.min = 0;
          if (chartRange[currentRound]) { //Set axis ranges if they exist
              if (chartRange[currentRound][1])
                  chartOptions.vAxis.viewWindow.max = chartRange[currentRound][1];
if (chartRange[currentRound][0])
                  chartOptions.vAxis.viewWindow.min = chartRange[currentRound][0];
}
          chart.draw(data, chartOptions);
      }
          chartOptions.hAxis.ticks = [];
          var rowCount = 1;
for(var i = 0; i < chartData[currentRound].length; i++) { //Add a chart row for each criterion if not null
              if (criteriaSelected[currentRound][i] && chartData[currentRound][i]) {
  data.addRow([rowCount, chartData[currentRound][i], barColors[0], (showLabels) ? chartData[currentRound][i].toFixed(1).toString() : ""]);
                  chartOptions.hAxis.ticks.push({v: rowCount++, f: (i+1).toString()});
              }
          }
          var data = new google.visualization.DataTable();
          data.addColumn('number', 'Criterion');
          var i;
for(i = 0; i < roundNum; i++) { //Add all columns for the data
              data.addColumn('number', 'Round ' + (i+1).toString());
              data.addColumn({type: 'string', role: 'style'}); //column for specifying the bar color
              data.addColumn({type: 'string', role: 'annotation'});
              var newRow = [];
              var elementsAdded = false;
              newRow.push(rowCount);
for(var j = 0; j < roundNum; j++) { //If the round has the criterion, add it
                  if (chartData[j][i]) {
                      newRow.push(chartData[j][i]);
                      elementsAdded = true;
                  } else {
                      newRow.push(null);
                  }
  newRow.push(barColors[j % barColors.length]); //Add column color
                  if (chartData[j] && chartData[j][i] && showLabels)
  newRow.push(chartData[j][i].toFixed(1).toString()); //Add column annotations
                  else
                      newRow.push("");
              }
      function loadCriteria() { //Creates the criteria check boxes
          var form = document.getElementById("chartCriteria");
while (form.firstChild) //Clear out the old check boxes
              form.removeChild(form.firstChild);
if (currentRound == -1) //Don't show criteria for 'all rounds'
              return;
          chartData[currentRound].forEach(function(dat, i) {
              var checkbox = document.createElement('input');
              checkbox.type = "checkbox";
              checkbox.id = "checkboxoption" + i;
checkbox.onclick = function() { //Register callback to toggle the criterion
                  checkboxUpdate(i);
              }
              var label = document.createElement('label')
    <form id="chartOptions" name="chartOptions">
      <select id="chartRounds" name="rounds"
onChange="updateChart(document.chartOptions.chartRounds.options[document.chartOptions.chartRounds.options.selectedIndex].value)" style = "display: none">
      </select>
<label><input type="checkbox" id = "labelCheck" checked="checked" style="display: none" onclick="showLabels = !showLabels; renderChart();">Show
Labels</label>
    </form>
    <div id="chartCriteria" name="chartCriteria">
    </div>


=== app/controllers/grades_controller.rb ===
=== Testing Issues ===
Using the Ubuntu image, one issue was discovered was with the ScoreView view. The migration files is building the view out as a table instead of a view as required, this can be seen in the schema file. The Development database shows the ScoreView as a view, but the Test database shows it as an table. It appears that the Development database was altered outside of migrations to force the ScoreView to be a view instead of table. With this scenario, the automated integration testing was forced to seed the ScoreView as a table. Further gems are available to alleviate this issue.


def action_allowed?
    case params[:action]
    when 'view_my_scores'
      ['Instructor',
        'Teaching Assistant',
        'Administrator',
        'Super-Administrator',
      'Student'].include? current_role_name and
        are_needed_authorizations_present?(params[:id], "reader", "reviewer") and
        check_self_review_status
when 'view_team'
      if ['Student'].include? current_role_name # students can only see the head map for their own team
        participant = AssignmentParticipant.find(params[:id])
        session[:user].id == participant.user_id
      else
true
      end
    else
      ['Instructor',
      'Teaching Assistant',
      'Administrator',
      'Super-Administrator'].include? current_role_name
    end
  end
  # collects the question text for display on the chart
  # Added as part of E1859
  def assign_chart_text
    @text = []
    (1..@assignment.num_review_rounds).to_a.each do |round|
      question = @questions[('review' + round.to_s).to_sym]
      @text[round - 1] = []
      next if question.nil?
      (0..(question.length - 1)).to_a.each do |q|
        @text[round - 1][q] = question[q].txt
      end
    end
  end
  # find the maximum and minimum scores for each questionnaire round
  # Added as part of E1859
  def assign_minmax(questionnaires)
    @minmax = []
    questionnaires.each do |questionnaire|
      next if questionnaire.symbol != :review
      round = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).first.used_in_round
      next if round.nil?
  @minmax[round - 1] = []
      @minmax[round - 1][0] = if !questionnaire.min_question_score.nil? and questionnaire.min_question_score < 0
                                questionnaire.min_question_score
                              else
                                0
                              end
      @minmax[round - 1][1] = if !questionnaire.max_question_score.nil?
                                questionnaire.max_question_score
                              else
                                5
                              end
    end
  end
# this method collects and averages all the review scores across teams
  # Added as part of E1859
  def assign_chart_data
    @rounds = @assignment.num_review_rounds
    @chartdata = []
    (1..@rounds).to_a.each do |round|
      @teams = AssignmentTeam.where(parent_id: @assignment.id)
      @teamids = []
      @result = []
      @responseids = []
      @scoreviews = []
      (0..(@teams.length - 1)).to_a.each do |t|
        @teamids[t] = @teams[t].id
        @result[t] = ResponseMap.find_by_sql ["SELECT id FROM response_maps
          WHERE type = 'ReviewResponseMap' AND reviewee_id = ?", @teamids[t]]
        @responseids[t] = []
        @scoreviews[t] = []
        (0..(@result[t].length - 1)).to_a.each do |r|
          @responseids[t][r] = Response.find_by_sql ["SELECT id FROM responses
            WHERE round = ? AND map_id = ?", round, @result[t][r]]
          @scoreviews[t][r] = Answer.where(response_id: @responseids[t][r][0]) unless @responseids[t][r].empty?
        end
      end
      @chartdata[round - 1] = []
      # because the nth first elements could be nil
      # iterate until a non-nil value is found or move to next round
      t = 0
      r = 0
t += 1 while @scoreviews[t].nil?
      while t < @scoreviews.length and @scoreviews[t][r].nil?
        if r < @scoreviews[t].length - 1
          r += 1
        else
def assign_chart_data
        end
      end
      next if t >= @scoreviews.length
(0..(@scoreviews[t][r].length - 1)).to_a.each do |q|
        sum = 0
        counter = 0
def retrieve_questions(questionnaires)
    questionnaires.each do |questionnaire|
      round = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).first.used_in_round
      questionnaire_symbol = if !round.nil?
                        (questionnaire.symbol.to_s + round.to_s).to_sym
                            else
                              questionnaire.symbol
                            end
      @questions[questionnaire_symbol] = questionnaire.questions
    end
  end
def update
    if format("%.2f", total_score) != params[:participant][:grade]
      participant.update_attribute(:grade, params[:participant][:grade])
      message = if participant.grade.nil?
          "The computed score will be used for " + participant.user.name + "."
                else
                  "A score of " + params[:participant][:grade] + "% has been saved for " + participant.user.name + "."
                end
    end
    flash[:note] = message
    redirect_to action: 'edit', id: params[:id]


=== app/views/grades/_team_charts.html.erb ===
== Shortcomings ==
<pre>
* We were unable to find two compatible assignments in the Expertiza test database to fully test our Compare Assignments feature. We did test all the functionality by seeding the database, but it technically wasn't using real data.
  <br> <br>
  <%= render :partial => 'criteria_charts' %>
  <br>


  <a href="#" name='team-chartLink' onClick="toggleElement('team-chart', 'stats');return false;">Hide stats</a>
  <br>
  <TR style ="background-color: white;" class="team" id="team-chart">
    <th>


    <div class="circle" id="average-score">
== Decisions ==
1. Put our new chart on the View Scores page, rather than on a new page
* Since it was able to fit on the existing page while displaying all the necessary info, we figured it'd be beneficial to have all grade statistics charts together.
2. Use GoogleCharts for our new charts
* The group who previously attempted this project chose GoogleCharts because it has a high compatibility and its charts look of similar quality to those already existing in Expertiza. We agreed with this, and determined GoogleCharts had all of the features we would need.
3. Added the AssignmentStats, CriterionStats, and ReviewRoundStats models
* This was a logical, structured way to organize the various data and methods we needed.
4. Encapsulation of the data within the new models
* Since the various statistics we needed to form our chart were stored in multiple models, there would be no good place to put the new methods we'd need. The way we did it promotes cohesion, since the new classes have the sole focus of computing statistics. This also allows for future refactoring of other sporadic statistics methods.
5. Placed AssignmentStats methods in assignment_stats_helper.rb rather than in the model or controller.
* The methods of interest were really not business logic, but more like massaging of data. For that reason it belonged more in the controller than the model. However, to keep things neat and clean in the controller, the methods were moved out to the helper file.
6. The exposure of mean and median as the metric_names in grades_controller.rb
* Since we are explicitly calling the avg_data and med_data methods in the controller, we thought it made sense to name the metric_names nearby, within the controller, rather than hidden off in the helper file.
7. Moved most of the javascript code in _team_charts_rubric_stats.html.erb into app/assets/javascripts/rubric_stats.js. We had to keep our dynamic references of Ruby code in the view.
* This keeps the view more clean and concise. Also, in the previous semester, one of the reasons the team's pull request was rejected was because they had too much javascript code in their views.


    </div>
      Class Average
    </th>
    <TH COLSPAN="8">
    <img src="<%= @average_chart  %>" ><br>
    Class Distribution
    <br>
    </th>
    <TH WIDTH="9">&nbsp;</th>
  </TR>


  <script type="text/javascript">
== Documentation ==
    var myCircle = Circles.create({
[https://github.com/expertiza/expertiza/pull/1440 Pull Request]
      id:          'average-score',
      radius:      50,
      value:        <%= @avg_of_avg.to_i %>,
      maxValue:    100,
      width:        15,
      text:        '<%=@avg_of_avg.to_i%>',
      colors:      ['#FFEB99', '#FFCC00'],
      duration:       700,
      textClass:      'circles-final'
    });
  </script>


  <style>
[https://github.com/gshugh/expertiza/tree/OSS-HW4 GitHub Repo]
  .circles-final{
      font-size: 16px !important;
    }
  </style>
</pre>


== Test Results ==
[https://drive.google.com/file/d/1jmOYfbNOlXQObtsK4MoIti-LsjJ_Y88P/view Demo Video]
The team ran both automated tests, using the RSpec framework, and manual tests of the user interface (UI), using the app. The automated tests helped to ensure that the basic functionality of the app still worked, while the UI tests ensured that the visualizations were correct.


=== RSpec Test Results ===
[https://docs.google.com/document/d/1Ozw2Bj2u_LyeKUdDqAyQtz1stEIKrxkmRnwmNyUgmzw/edit#heading=h.5kwrelqdnewh Problem Statement]
In order to ensure that our changes did not affect basic functionality of the app, we made sure that all tests that passed before our changes also passed after our changes. Add statistics on the tests before our changes and matching statistics after our changes.


=== UI Tests ===
To validate all functionality of the chart when adding new features or fixing old ones, the following criteria were tested manually for expected functionality:


# Chart is displaying correctly
== Team ==
## Bars are showing up where expected
* Akshay Ravichandran '''(Mentor)'''
## Bar annotations are showing the expected value
* Chris Adkins
## Criteria labels are for the correct bar and displaying correct values
* Bobby DeMarco
## Hover text is displaying the correct values
* George Hugh
## Null values are not present on the chart
* John Warren
## Correct colors are used for the multi-round view
# Show Labels checkbox works as expected
# Round Criteria is displaying correctly
## Round dropdown menu shows all rounds for the assignment
## Selecting a round changes the criteria checkboxes
## All checkboxes are displayed with appropriate text
## Checkboxes correctly remove or add criterion bars to the chart

Latest revision as of 03:21, 3 May 2019

Introduction

This Expertiza project, completed in the Spring of 2019, aimed to improve the charts instructors could view to see the grade statistics for a given assignment. Two charts already existed, one showing the grade distribution for all teams, and another simply showing the class average grade. Our goal for this project was to add a third chart, using Google Charts, which would show grade statistics for the various rubric grades within that assignment. The chart is interactive so the user (instructor) can toggle which rubric criteria, and which statistics, to display.


Along with displaying the rubric criteria for a single assignment, we aimed to add a feature which would allow the comparison of compatible rubric grades from two different assignments. Again, this functionality is interactive, allowing the instructor to choose which statistics will be populated in the chart.


By viewing our new visualization of rubric grades, an instructor can better judge which aspects of the course are well-understood, and which may need a bit more attention.



Problem Statement

E1929 - Visualizations for Instructors - Class performance on specific rubrics

An interactive visualization or table that shows how a class performed on selected rubric criteria would be immensely helpful. It would show the instructor what he / she will need to focus more attention on. For example, could you create a graph showing the range and clustering of scores for the 5 main rubric criteria? And, if these same 5 criteria are used in the preliminary and final assignments, it would be nice to be able to compare performance between assignments in a visualization that showed the class results on 3 separate artifacts.

Webpage: login as instructor -> Manage -> Assignments -> View scores


Old Charts

Figure 1: Existing assignment grade charts - The existing assignment grade charts show the average grade for the assignment and the grade distribution. During this project, we aimed to add a third chart which shows the distribution for specific rubric criteria within the assignment.


Accomplished

Executed on Plans

We successfully executed all of the ideas we had during the planning phase:

  • Add new feature to show mean and median data for rubric criteria in a given assignment
  • Add new feature to show comparison
  • Performed UI testing to ensure features were operational, including edge cases
  • Added and ran RSpec and Capybara automated tests


New Charts Inserted

Figure 2: New Rubric Statistic Visualization - Added in the center of the existing two charts is an interactive chart the instructor can use to see the mean or median scores for whichever criteria they select. This is done on the "Analyze Assignment" tab, which is selected by default when the "View Scores" page is loaded.


Figure 3: New Rubric Cross-Assignment Comparison Visualization - The instructor can see a comparison between selected criteria between the current assignment and the selected assignment. The rubric criteria must be compatible in order for it to even show up in the selection list. This comparison visualization is done on the "Compare Assignments" tab which must be selected after the "View Scores" page has been loaded.


Extra Issues Fixed


Code Changes

Files Added/Modified

Added: 10 + Modified: 11 = Total: 21

  • app/assets/javascripts/rubric_stats.js (new)
  • app/controllers/grades_controller.rb
  • app/helpers/assignment_stats_helper.rb (new)
  • app/models/assignment.rb
  • app/models/assignment_questionnaire.rb
  • app/models/assignment_stats.rb (new)
  • app/models/criterion_stats.rb (new)
  • app/models/question.rb
  • app/models/questionnaire.rb
  • app/models/review_round_stats.rb (new)
  • app/models/score_view.rb
  • app/views/grades/_team_charts.html.erb
  • app/views/grades/_team_charts_averages.html.erb (new)
  • app/views/grades/_team_charts_distribution.html.erb (new)
  • app/views/grades/_team_charts_rubric_stats.html.erb (new)
  • spec/controllers/grades_controller_spec.rb
  • spec/factories/factories.rb
  • spec/features/grade_interface_spec.rb (new)
  • spec/features/helpers/grade_interface_helper.rb (new)
  • spec/models/assignment_spec.rb
  • spec/models/questionnaire_spec.rb


Testing

We performed automated tests using RSpec Framework and Capybara. In addition, we performed manual tests of the user interface (UI), by using the app.

RSpec Framework Tests

The RSpec Testing Framework, automated testing, was used to verify the models of the Expertiza web application feature set. Since this feature is dealing with visualizations (charts) that are intimately tied with Active Record models, we seeded the testing database with known data via FactoryBot gem. These changes were automatically rolled-back once the testing was complete.

RSpec was used to increase/maintain automated test coverage for the methods that we added into assignment and questionnaire models. Mocks and stubs were utilized in order to decouple the model under test from dependencies of other models. This will allow us to identify easily if our model updates are causing an issue.

Capybara Tests

RSpec Capybara was used to provide integration testing of the charts. The Test Database was seeded with the appropriate information for executing the results of the charts. An additional GEM was needed, webdrivers, to utilize the predefined drivers of Selenium. The default web driver, Rack-Test, does not execute java scripts. Since the graphs that we are using require java script, thus soliciting the necessity of Selenium with its predefined web driver, selenium_chrome_headless. This provides the required functionality to process the java scripts and to automatically ensure that they exist in the DOM.

In order to use the headless web driver with the seeded Test Database, a different database strategy was required. As a result, for this test only, the database strategy was changed to truncation. Ruby 2.2.7 is used in the current version of Expertiza. This version does not allow for sharing of the database thread, and thus requires the to be written to the database outside of a transaction block. This allows for the headless web driver to see that the data is persisted the Test Database and is able to run as normal. Once the test is completed, the database tables are truncated and all data is removed.

UI Tests

In addition to the automated tests above we also performed manual testing of the newly added features to include:

  • The new feature properly initializes
    1. Analyze Assignment tab is loaded by default
    2. Chart is in between the other two existing charts
    3. Round 1, all Criteria selected, and Mean are selected by default
  • Analyze Assignment tab operating correctly
    1. Bars are showing up where expected
    2. Bar annotations are showing the expected value
    3. Criteria labels are for the correct bar and displaying correct values
    4. Hover text is displaying the correct values
    5. Null values are not present on the chart
    6. Round dropdown menu shows all rounds for the assignment
    7. Selecting a round changes the criteria checkboxes
    8. All checkboxes are displayed with appropriate text
    9. Checkboxes correctly remove or add criterion bars to the chart
  • Compare Assignments tab operating correctly
    1. Compare Assignments tab is only displayed if compatible assignments exist
    2. Clicking Compare Assignments tab loads that tab
    3. (All tests from Analyze Assignment tab apply to this tab as well)
    4. The two colors for assignment comparison are correct

Testing Issues

Using the Ubuntu image, one issue was discovered was with the ScoreView view. The migration files is building the view out as a table instead of a view as required, this can be seen in the schema file. The Development database shows the ScoreView as a view, but the Test database shows it as an table. It appears that the Development database was altered outside of migrations to force the ScoreView to be a view instead of table. With this scenario, the automated integration testing was forced to seed the ScoreView as a table. Further gems are available to alleviate this issue.


Shortcomings

  • We were unable to find two compatible assignments in the Expertiza test database to fully test our Compare Assignments feature. We did test all the functionality by seeding the database, but it technically wasn't using real data.


Decisions

1. Put our new chart on the View Scores page, rather than on a new page

  • Since it was able to fit on the existing page while displaying all the necessary info, we figured it'd be beneficial to have all grade statistics charts together.

2. Use GoogleCharts for our new charts

  • The group who previously attempted this project chose GoogleCharts because it has a high compatibility and its charts look of similar quality to those already existing in Expertiza. We agreed with this, and determined GoogleCharts had all of the features we would need.

3. Added the AssignmentStats, CriterionStats, and ReviewRoundStats models

  • This was a logical, structured way to organize the various data and methods we needed.

4. Encapsulation of the data within the new models

  • Since the various statistics we needed to form our chart were stored in multiple models, there would be no good place to put the new methods we'd need. The way we did it promotes cohesion, since the new classes have the sole focus of computing statistics. This also allows for future refactoring of other sporadic statistics methods.

5. Placed AssignmentStats methods in assignment_stats_helper.rb rather than in the model or controller.

  • The methods of interest were really not business logic, but more like massaging of data. For that reason it belonged more in the controller than the model. However, to keep things neat and clean in the controller, the methods were moved out to the helper file.

6. The exposure of mean and median as the metric_names in grades_controller.rb

  • Since we are explicitly calling the avg_data and med_data methods in the controller, we thought it made sense to name the metric_names nearby, within the controller, rather than hidden off in the helper file.

7. Moved most of the javascript code in _team_charts_rubric_stats.html.erb into app/assets/javascripts/rubric_stats.js. We had to keep our dynamic references of Ruby code in the view.

  • This keeps the view more clean and concise. Also, in the previous semester, one of the reasons the team's pull request was rejected was because they had too much javascript code in their views.


Documentation

Pull Request

GitHub Repo

Demo Video

Problem Statement


Team

  • Akshay Ravichandran (Mentor)
  • Chris Adkins
  • Bobby DeMarco
  • George Hugh
  • John Warren
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=E1929_Visualizations_for_Instructors&oldid=124828"