CSC/ECE 517 Fall 2017/E1792 OSS Visualizations for instructors

This wiki page is for the description of changes made under E1792 OSS Visualizations for instructors.

About Expertiza

Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Instructors can also use Expertiza for interactive views of class performances and reviews.

Introduction

This project aims to improve the visualizations of certain pages related to reviews and feedback in Expertiza in the instructor's view. This would aid the instructors to judge outcomes of reviews and class performance in assignments via graphs and tables, which in turn would ease the process of grading the reviews.

Problem Statement

• Issue 1: The scale is blue to green, which does not make sense. And colors will change randomly each time loading the page. It will be better to scale from red to green.
• Issue 2: Two adjacent bar represents the response in round 1 to round k. It makes sense only if the rubrics in all review rounds are all the same. If the instructor implements the vary-rubric-by-round mechanism, this visualization will not make sense.
• Issue 3:The table is presorted by teams in the page View Scores, but you can now also sort alphabetically. The cell view looks way too long, and should be divided into partials.
• Issue 4: An interactive visualization or table that shows how a class performed on selected rubric criteria would be immensely helpful. It would show the instructors what they need to focus more attention on.

UML Diagram

Solutions Implemented

Issue 1

Description:The scale is blue to green, which does not make sense. And colors will change randomly each time loading the page. It will be better to scale from red to green.

Approach: Here the color coding is blue to green and also it changes randomly which one color in one graph may represent some other score in the next graph. To fix this issue, we have redefined the highchart_colors data structure to contain the colors defined in the grades.scss file (already defined earlier) as suggested by the Professor. This way no matter what scale is being used it will always have appropriate color coding. Also, the color legend is now appropriate to explain correctly the color-code used.

highchart_colors = []
   highchart_colors.push("#2DE636")
   highchart_colors.push("#BCED91")
   highchart_colors.push("#FFEC8B")
   highchart_colors.push("#FD992D")
   highchart_colors.push("#ff8080")
   highchart_colors.push("#FD422D")

Related screenshot

Issue 2

Description: Two adjacent bar represents the response in round 1 to round k. It makes sense only if the rubrics in all review rounds are all the same. If the instructor implements the vary-rubric-by-round mechanism, this visualization will not make sense.

Approach: Here The problem is that the the review responses show in a grid format that shows all the reviews from round 1 to round n. But this is ok when the rubrics is same for all the rounds. But when the rubric is different from round to round this view doesn't make sense. So what we are thinking is creating different grid for each review round. That will solve the problem for when the rubric is different as well as when the rubric is same.

def get_highchart_data(team_data, assignment, min, max, number_of_review_questions)
   chart_data = {}  # @chart_data is supposed to hold the general information for creating the highchart stack charts
   for i in 1..number_of_review_questions
     chart_data[i] = Hash[(min..max).map {|score| [score, Array.new(assignment.rounds_of_reviews, 0)] }]
   end
   team_data.each do |team|
     team.each do |vm|
       next if vm.round.nil?
       j = 1
       vm.list_of_rows.each do |row|
         row.score_row.each do |score|
           unless score.score_value.nil?
             #chart_data[vm.round][score.score_value][j] += 1
             chart_data[j][score.score_value][vm.round-1] += 1
           end
         end
         j += 1
       end
     end
   end
   chart_data
 end

 @list_of_rows = []
   @list_of_reviewers = []
   @list_of_reviews = []
   @list_of_team_participants = []
   @max_score = questionnaire.max_question_score
   @questionnaire_type = questionnaire.type
   @questionnaire_display_type = questionnaire.display_type
   @rounds = rounds
   @round = round
   @name  = questionnaire.name

 def generate_highchart(chart_data, min, max, number_of_review_questions, assignment, team_data)
   # Here we actually build the 'series' array which will be used directly in the highchart Object in the _team_charts view file
   # This array holds the actual data of our chart with legend name
   highchart_series_data = []
  
   count_rounds=0
   chart_data.each do |round, scores|
     scores.to_a.reverse.to_h.each do |score, rubric_distribution|
       if count_rounds == 0
         highchart_series_data.push(name: "Score #{score}", data: rubric_distribution, stack: "S#{round}")
       else
         highchart_series_data.push(linkedTo: "previous",name: "Rubric #{round} - Score #{score}", data: rubric_distribution, stack: "S#{round}")
       end
     end
     count_rounds = count_rounds + 1
   end

   # Here we dynamically creates the categories which will be used later in the highchart Object
   highchart_categories = []
   for i in 1..@assignment.rounds_of_reviews
     highchart_categories.push("Submission #{i}")
   end

   # Here we dynamically create an array of the colors which the highchart uses to show the stack charts.
   # Currently we create 6 different colors based on the assumption that we always have scores from 0 to 5.
   
   highchart_colors = []
   highchart_colors.push("#2DE636")
   highchart_colors.push("#BCED91")
   highchart_colors.push("#FFEC8B")
   highchart_colors.push("#FD992D")
   highchart_colors.push("#ff8080")
   highchart_colors.push("#FD422D")

   [highchart_series_data, highchart_categories, highchart_colors]
 end

Related screenshot

Issue 3

Description: The table is presorted by teams, but you can now also sort alphabetically. The cell view looks way too long, and should be divided into partials.

Approach: Here we can sort the select query with appropriate column alphabetically, or we can sort the table automatically according to user criteria using dynamic table format. And the long view issue can be solved by using paging.

Related screenshot

Issue 4

Description: An interactive visualization or table that shows how a class performed on selected rubric criteria would be immensely helpful. It would show me what I need to focus more attention on.

Approach: The issue is about getting the performance of the entire class graded using the 5 rubrics criteria.Scores will be color coded for each of the rubrics in each of the submissions. When hovered over the graph , for each score, the instructor should see the number of students , the percentage of students that has scored that particular point in that rubric in that submission.

Related screenshot

Test Plan

Issue 1:Check if color coding from red to green for a range of score

• 1) Login as instructor
• 2) Create a dummy assignment and come dummy reviews for the same.Log out.
• 3) Login as a student. Attempt the assignment. Log out.
• 4) Login as another student and repeat step 3.
• 5) Login as either student and attempt the review. Logout.
• 6) Login as instructor. Go to Review grades page and check the table. If color code ranges from red ( for least score) to green ( for highest score), then test passed.

Issue 2:Check if different graph visible for different review submission

• 1) Login as instructor.
• 2) Create an assignment. Select 2 reviews. Also select different rubrics for both reviews.
• 3) Login as a student ("A") and submit the assignment. Repeat this for another student("B").
• 4) Login as student A and perform review. Do this for student B too.
• 5) Now resubmit assignment as student A and B again.
• 6) Resubmit reviews as student A and B again. This time the rubrics will be different from the previous round.
• 7) Now login as instructor and see the visualization of the reviews. You can see the different graphs for different submissions.

Issue 3:Check if the table is sorted with appropriate column alphabetically

• 1) Login as instructor
• 2) Create a dummy assignment with some teams. Logout.
• 3) Login as a student and attempt the assignment and logout.
• 4) Repeat step 3 for all dummy teams.
• 5) Login as instructor.
• 6) Go to View Scores page. Check the grade table.
• 7) Click on a column header and check if data in it is getting sorted alphabetically. If yes, then the test passed.

Issue 4:Check if graph in Issue 3 shows how a class performed on selected rubric criteria

• 1) Login as the instructor
• 2) Click on the button to compute graphs
• 3) Compare the bar graphs with separate scores of students in each assignments.

References

• Expertiza repo
• The live Expertiza website
• Highcharts API