CSC/ECE 517 Fall 2016 E1709 Visualizations for instructors

Introduction

Expertiza is an open-source web application to create re-usable learning objects through peer-reviews to facilitate incremental learning. Students can submit learning objects such as articles, wiki pages, repository links and with the help of peer reviews, improve them. The project has been developed using the Ruby on Rails framework and is supported by the National Science Foundation.

Project Description

Purpose and Scope

Expertiza assignments are based on a peer review system where the instructor creates rubrics for an assignment through questionnaires which students use to review other students' submissions. The author of the submission is given an opportunity to provide feedback about these reviews. All questions in the questionnaire have an assigned grade which is based on a pre-defined grading scale. Instructors can see a report on the scores of a student given by reviewers, on the score of feedback given to the reviewers and many other reports. Based on these reports (which are all on separate pages), the instructors grade the student. These reports are, however, on separate pages and it is difficult for an instructor to navigate to many pages before grading the student. The first requirement is meant to solve this problem by merging the review scores with the author feedbacks on the same page.

Furthermore, there is no way for an instructor to evaluate the rubric he/she has posted on assignments. Students may consistently be getting negative reviews for a rubric which might not be totally relevant to the assignment thereby reducing their scores. A report of average class scores for each rubric in the questionnaires would help instructors refactor their rubric and understand where students generally perform well and where they struggle. This new report forms the second part of the requirement.

We are not modifying any of the existing functionalities of Expertiza. Our work would involve modifying the review report and creating a new report for average class scores.

Task Description

The project requires completion of the following tasks:

• Integrate review data with author feedback data to help instructors grade the reviewers.
• Add a new column to give the length of reviews.
• Separate data from columns number of reviews and summary details.
• Create a new column to check if a reviewer added a link or a file.
• Create a new column to check the average author feedback rating per team.
• Add interactive visualization to show class performance of an assignment to an instructor.
• Create a new route/view/controller for class performance.
• Add a new link to point to the new controller created. This new link will be created per assignment.
• Create two new views, one for selecting rubric criteria and second to show the graph.
• Create graphs to show the class performance as per the rubric metrics selected dynamically.

Project Design

Design Patterns

Iterator Pattern<ref>https://en.wikipedia.org/wiki/Iterator_pattern</ref>: The iterator design pattern uses an iterator to traverse a container and access its elements. When we are implementing the response and author feedback report, we will be iterating through each reviewer to get the review performed by them and then each author based on the feedback given for each review. This iteration will occur with the data returned by the ResponseMap model for all of the review and feedback information. For the class performance report, we will be iterating through each questionnaire per assignment, and thereafter each question per questionnaire. The same iteration will also be required to get answers per question per reviewer.

Review and Author Feedback Report

Workflow

Class Performance Report

Workflow

Implementation Thoughts

In order to implement the above functionality for the class performance report, our initial observation resulted in the following path for implementation.

• We will need a new controller ClassPerformanceController.
• We will need the following two new views.
  • A view to select_rubrics. This view will allow instructors to select a couple of rubrics to evaluate the class performance on.
  • A view to show_class_performance. This view will display the class performance using relevant graphs to represent the information clearly.
• Routes for each of the views created.
• Model methods to facilitate getting the required information and calculations.

We need to provide a link to the instructor to see this view. As shown above, this will be a button in the assignment management page routed at tree_display/list which corresponds to the function list in the controller TreeDisplayController.rb. The button we are adding here to route to the newly created select_rubrics view.

The select_rubrics view will receive the following parameters.

assignment_id

The controller will get all the questionnaires related to that assignment from the AssignmentQuestionnaire model. It will then get a list of all the rubrics used in those questionnaires from the Questions model. These will be displayed to the instructor. It will then route the instructor to the show_class_performance view upon selection of rubrics. It will pass the following parameters to the show_class_performance view.

assignment_id
Array[question_id]

The controller will take the list of questions for the assignment and find all the answers for those questions. It will then calculate the average score per question for the entire class from those answers and pass this to the view. The show_class_performance view will use the Gruff API in order to provide an aesthetically appealing visualization of the data.

Use Cases

• View Review and Author Feedback Report as Instructor: As an instructor, he can see the different metrics of reviews and average feedback rating received per student done for an assignment or a project.
• View Class Performance as Instructor: As an instructor, he can select 5 rubric metrics used per assignment. The instructor is able to see the graph to check the class performance based upon the metrics selected.

Test Plan

• For use case 1, test if the instructor can see the text metrics of reviews and author feedbacks received for an assignment or a project per student.
• For use case 2, test if the instructor can select 5 rubric metrics used for an assignment. Also, test if the instructor can view the class performance from a graph using the metrics selected by the instructor.

System Requirements

Hardware Requirements

• Computing Power: Same as the current Expertiza system.
• Memory: Same as the current Expertiza system.
• Disk Storage: Same as the current Expertiza system.
• Peripherals: Same as the current Expertiza system.
• Network: Same as the current Expertiza system.

Software Requirements

• Operating system environment: Same as the current Expertiza system.
• Networking environment: Same as the current Expertiza system.
• Tools: Gruff API in addition to the current Expertiza system.

References

<references/>