CSC/ECE 517 Fall 2017/E1789 Semester Project

Improvements to review grader

1. Overview

In expertiza, review report page and summary page give the information for instructors to view the overall review result. Also, they are the place where instructors grade the reviews. Review report page can be accessed via Manager -> Assignment, then click the “View review report” button (the one which has a spyglass and two people). The summary can be accessed through Review Page by clicking “summary” button of a particular student.

The appearances of the two pages are shown below:

For now, there are several defects need to be fixed to optimize the grade procedure:

• Metrics column is not intuitive enough.
• The layout of “Review Report Form” in “Summary Page” is unreasonable.
• Reviewers don’t need to re-review the submissions which are not updated, so there should be a checking mechanism.

Therefore, our tasks are:

• Visualize Metrics column.
• Reasonably reorganize the “Review Report Form” in “Summary Page”.
• Check updates of submissions for reviewers to help them decide if they need to review aga.

2. Visualize Metrics column

2.1. Summary of problems

Metrics column, which is in “Review Report Form”, displays statistic data of words used in particular student’s reviews. This helps graders to value how the student compares with the average student. For now, it displays numerical data in the literal way which is not intuitive. It would be better to have a bar or column chart showing this. Ultimately, we may be showing several metrics in this column, so the bars or charts should be resizable.

2.2. Chart design in the metrics

We compare the 1st round and the 2nd round review with their average separately, then compare the overall (Total) with the average. In this way, it will be pretty clear for the grader to checkout how the student performs in the reviews.

3. Reorganize "Review Report Form"

3.1. Summary of problems

• There is no header saying what course, assignment, or student this relates to.
• The team name and student names are listed on separate rows. With the large amounts of whitespace, this makes the table too sparse vertically. It also takes up too much space as the other columns usually contain more information.
• Text is too close to the cell boundaries.

• For checkbox questions, comments are not possible, but still a blank cell is displayed.
• Checkbox questions could be displayed more compactly, better in a visually appealing manner like the review is shown to the author.

3.2. Header in summary page

It will be more reasonable if there is a header in the page. The information of the header will include course name, project name and student number as below:

For example, the header will show as “CSC 517 OSS PROJECT : Student 666”

3.3. Form design in summary page

We totally redesign the form in order to arrange the space more properly. The form looks as below：

Firstly, we lay the teams and questions into the horizontal and vertical columns respectively, so that we could avoid the problem of listing the team repeatedly for each question in the previous way. At the same time, it solves the problem that the team field takes up too much space.

Secondly, for questions that require score and comment, we show the score at the beginning of the cell, then the text follows. This is consistent with the way that review is shown to the author.

Finally, for the checkbox, it will become easy. The result will show as ✔ or ✘ that replaces the previous manner of using 0 or 1. This also removes the previous comment column.

4. Check updates for reviewer

4.1. Summary of problems

In the review report, each team that has been reviewed is color-code. Text in red indicates that the review is not yet completed; text in blue indicates that the review grade is not assigned or updated. So if an instructor sees a name in red, the student should not be given credit for the review. If text is in blue, the instructor should grade it now. But another common case is after the reviewer has reviewed the work, the author didn’t update it. So the reviewer doesn't need to re-review the work. These reviews should be coded with another color (perhaps green). Therefore, it should be checked whether the author has submitted new files or links since the previous review, even the content pointed to by pre-existing links.

4.2. Add checking mechanism

To learn whether submission has been updated since the previous review, we need to get the latest updated time then compare it with the latest review time. In expertiza, one submission contains two types of content -- link and file. Both two types should be considered. For file, we just need to look at the last modification time. For link, you need to consider more: the link string itself, and the content that the link points to.

4.2.1. Check update of file and link string

In expertiza, there is a SubmissionRecord class that stores the records of each submission. Thus, for the file and the link string, we could simply get the update time from the records.

4.2.2. Check update of link-to content

For link-to content, situation becomes much more complicated. The types of the links are various, and how to get update times from them is what we need to focus on.

Actually, Many HTTP responses return a Last-Modified header. It contains the date and time at which the origin server believes the resource was last modified. Sometimes we could use this time as our latest update time.

In other cases, we can’t get Last-Modified header from the URL. Like the GitHub Enterprise, generally it requires you to login to access the resources. Even you have logged in, the Last-Modified time is not equal to the latest commit time which is what we want. In these cases, we need to call APIs of the link-to website. As the GitHub and Google Doc should be the most common websites, we will handle these two special types of links.

4.3. Mark not updated review

After we got the status of submission(updated or not), we could simply mark the not updated reviews green to indicate that it doesn't need to be re-reviewed.

5. Test Plan

The project is about change the UI Style and improving user experience. It does not add any functionality to Expertiza. So there is no scope for testing in this project.