CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

E1777. OSS Project Green: Coherent specification of review requirements

This page provides a description of the Expertiza based OSS project.

About Expertiza

Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

Problem Statement

The following tasks were accomplished in this project:

• Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
• Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
• Handled issue of project submission assignment process to ensure every submission receives required number of reviews.

Implementation

Functionality

• In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
• The Review Strategy tab includes options, "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer."
• The capitalization and punctuation in the statements ***** under the review strategy tab are random.
• On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the users. This means that if reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed by anyone else.

Drawbacks and Solutions

• Problem 1: The system always allocates to a reviewer, that submission which has received a minimum number of reviews. If a reviewer has already reviewed the submission with the minimum number of reviews and he requests for a new submission to review, he does not receive any submission to review.

• Solution: This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
• Problem 2: The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

• Solution: Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
• Problem 3: The capitalization and punctuations of statements **** are incorrect.
• Solution: This view has been fixed by making changes in the respective files.
• Problem 4: the assignment of a submission to a reviewer depends on the number of reviews it has previously received, if that submission has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same submission to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where a submission was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
• Solution: This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for a submission so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

References

1. Expertiza on GitHub
2. GitHub Project Repository Fork
3. The live Expertiza website
4. Demo link
5. Expertiza project documentation wiki
6. Rspec Documentation
7. Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin