CSC/ECE 517 Spring 2018- Project E1816: Visualization for Instructors
Team Members
Daniel Burcal (djburcal@ncsu.edu)
Patrick Poggi (pfpoggi@ncsu.edu)
Ambareesh Pandit (aspandit@ncsu.edu)
Minghao Pan (mpan2@ncsu.edu)
Expertiza
Expertiza is an open source web application based on Ruby on Rails framework, supported by the National Science Foundation. It is a place where students can submit and peer-review learning objects (assignments, codes, write ups, websites, etc). For an instructor, expertiza allows to create and customize new or existing assignments. For students, it allows to create and work on various projects and assignments. It provides a platform to peer review other students' submissions across various document types, including the URLs and wiki pages.
Problem Statement
A previous team has worked on creating this visualization graphic for Instructors, which can be found here. We are tasked with building upon the work they have performed as well as improving the visualization aspect for instructors. Specifically we can narrow the goals of this project as follows:
- The Visualization of the compiled metrics is confusing. We will aim to use a more coherent color scheme from red to green for the graph. We want to allow the instructor to dynamically generate graphs with the data they request. We will separate the table into partials or use a scheme to only show the sections of the table the instructor requests
- There are ways to compile review data together such that it is even easier for the instructor to identify weaknesses and strengths. To that end, we would like to integrate review performance by compiling various feedback data that is collected. This would include the number of reviews performed, the length of the reviews, the summary of the reviews, whether or not the reviewers uploaded a file in their review, the average ratings the reviewers received from authors.
- Another useful feature would be the ability to visualize class performance on a certain rubric criteria. Further, if these criteria are common ones used between projects, it would be very helpful to be able to compare performance between assignments on the common criteria.
Design Process Flow
Below you can see a flowchart representing the graphical flow of an instructor visiting the visualization page we are modifying.
Once the instructor visits the visualization page, they will already see a bar graph displaying how all the teams scored on each rubric criteria. Below this will be buttons or some other method by which to select the data that you want. The whole process flow involves being able to traverse the visualization page in a quick and efficient manner, since oftentimes the instructor will visit the page often. Below we will go into our intended design route for this project.
Design
Here is a mock-up of our intended page:
As you can see, we will have menu options by which the instructor can choose which data members to show and likewise the type of graphic visual they would like to produce. This will make it so that the instructors can visualize only the data they want to, and in the form that gives them the most relevant information. We plan on using the flot javascript library to implement the graphic visuals, which will provide an easy and clear visual of the requested data. Under the visual, we will have a way to select between teams and individual students' data to further target the desired visual. The last team which worked on this project used the highcharts javascript library, but we have decided to move to flot due to visual clarity and the ability to easily decide which data to display.
Another mock-up of how the UI could be designed, is that by visiting the visualization page, the instructor will see a graph followed by an array of buttons.
The chart above shows stacked bar charts of a single submission. Each bar represents a question from the submission and the colors represent the percentage of teams that earned a 0(red) through 5(green) on the submission.
As you can see, we plan on implementing an array of buttons to carry out simple manipulations to data and to display only the data the instructor wants to. We plan on implementing buttons to change the graphic that is displayed, so that if an instructor wishes to see a pie chart or histogram, they can click the button and see the data in a different form. Below those buttons, there will be other ones which will direct the page on what type of table to display and with which information. The reason for these adjustments are that in the last semester's implementation, the table was quite lengthy, and this will make it more manageable by the instructor.
It is possible that if we are able to, we plan on implementing a drop down functionality like in the mock-up further up the page where you can be even more selective about the data you wish to display in the tables or in the graphics.
Modified Files
Generally we plan on modifying the same files as the previous group did. The controllers of grade and assignment are involved. For the view part, we will modify the view of grade and review_mapping. We may also create new view files. app/controllers/grades_controller.rb
app/controllers/assignments_controller.rb
app/views/grades/view.html.erb (partials below)
app/views/grades/_teams.html.erb
app/views/grades/_team_title.html.erb
app/views/grades/_team_charts.html.erb
app/view/review_mapping/_review_report.html.erb
Testing Plan
Since it is based on a previous project, we need to refactor some previous code while making sure the previous test files (assignments_controller_spec.rb and grades_controller_spec.rb) still work. We are going to test new features we add in the future.
It is beyond the scope of the course to test the graphs. We will only test the Ruby code.The tests will most likely use RSpec and the existing factories in Expertiza.
As we will use Javascript to achieve an interactive visualization, we will test the functionality of Javascript codes manually.
Implementation
Visualization
In order to Improve the visualization of the rubric question data, we decided to use the flot javascript library instead of highcharts. In order to accomplish this, we constructed an adapter that took the already generate highchart data by the previous team, and converted it to a representation required by our flot graph. This is done in the highchart_to_flot_adapter method in the app/controllers/grades_controller.rb file.
Color Scheme
For fixing the color scheme, we used the adapter that was made to obtain the correct data, and instead of using their randomly generated colors, in the adapter there was a 6 element array with colors from red to green. This was inputted into the data series as per the correct format flot requires. You can also see the result below, in the picture of the graph provided.
Rubric Representation
To implement this, in the adapter we formed the data series such that different review rounds would be separated and clustered among their own review questions. This allows an easier viewing of different review rounds in order to determine how the class improved. The data passed to the _team_charts.html.erb view file contains all of the questions for all review rounds associated with the assignment. In addition to implementing the flot graphical representation, we also created a tooltip which allows the instructor to see the percentage of people who scored a certain value on any rubric questions of any round.
Here is a picture of what the completed flot graph looks like:
Class Performance
The Class Performance visualization allows the instructor to view a histogram of 0 to 5 scores for a particular assignment. Furthermore, the visualization allows the user to switch between the preliminary round and the final round as well as view a combined view. As with the rubric criteria visualization, hovering over a bar in the class performance visualization brings up a tooltip indicating the score and frequency of teams that earned that score.
Review Performance
There some obsolete codes in review_mapping_helper.rb about Dr. Kidd's course. Those codes were intended to calculate author feedbacks of reviews and there are also some related codes in _review_report.html.erb. However, in the view part, those codes are commented and those do not work. In fact, Dr. Kidd's course is 806, which is not on Expertiza now. So those obsolete codes are removed. Since in the testing files there are no corresponding codes for those parts, the previous tests still work. The new review table allows the instructor to sort the result by more options, like the number of reviews completed, the average volume and the average score. The page is given as below.
The remaining challenge is to integrate the author feedback to the review report page. It is hard because of multiple reasons. Some reviews have 3 rounds. The feedback of each round has multiple questions. Some are scored questions but some are not. In fact, even the author feedback report does not include the specific score.
Testing Done
Rubric Representation
In terms of ensuring correct operation of the system under the changes made, our goal was to ensure the previous tests worked, as not much was entirely changed about the view controller method. In ensuring the previous tests worked, we could then be sure the private methods within the grades_controller.rb file were working properly.
Class Performance
The tests verify that the correct frequency of raw scores is received from the database. A test is performed for rubric criteria that varies by round and one is created for an assignment where the rubric does not vary.
Future Work
One consideration to improve upon the work we have done, is to make it so that the visualization graph that shows participant performance on every rubric question in each round can handle assignments which vary the number of rubric questions between rounds. Currently the set-up does not support such a feature.
With regards to the class performance, an enhancement of showing the breakdown of weighted scores 0-100% could be shown instead of displaying the frequency of raw 0-5 scores.
References
Here is our project repo
Here is our created Pull Request
Here is our video Visualization for Instructors Walkthrough