CSC/ECE 517 Spring 2018/E1814 Write unit tests for collusion cycle.rb

Introduction

Team

Zhewei Hu (zhu6) (mentor)

• Cameron Nelson (cenelso3)
• Peter Shore (pnshore)
• Robert Montgomery (rjmontgo)

Task

Our task for this project is to test the collusion_cycle.rb model with the collusion_cycle_spec.rb test to achieve at least ninety percent coverage.

Background

Expertiza

Expertiza is an open source project that can be used by instructors to create assignments and assign them to students. Students can manage teams for the assignment and submit work for each of the assignments. After the students have submitted their work they can review other student's work to facilitate grading. The project is based on the Ruby on Rails framework. You can find our GitHub Repo as well as our pull request to view our work.

Collusion Cycles

The collusion cycle model is used to handle situations where a cycle is created when students are reviewing each other's work. The cycles in the review relationships are common since each assignment to review is randomly selected. The collusion cycle model calculates the assignment scores from the reviewers as well as other statistics such as similarity and difference between each score. The model is built to handle two, three, and four node cycles where each node represents a student.

Motivation

Achieving good coverage with quality tests is part of a good development process. The collusion_cycle.rb class is used to calculate the grade received based on the reviews from other students as well as handle cases where the reviewer is also reviewed by the reviewee effectively creating a cycle. We want to test this class to achieve good coverage in order to catch possible bugs in the program and ensure that it works correctly.

Test Plan

Tasks Identified

• Test Two-Node Cycle
  1. When the reviewers of the current reviewer (ap) do not include the current assignment participant it should skip this reviewer (ap) and return the corresponding collusion cycle
  2. When the reviewers of the current reviewer (ap) include the current assignment participant and the assignment participant is not reviewed by the current reviewer it should skip the reviewer (ap) and return the corresponding collusion cycle.
  3. When the reviewers of the current reviewer (ap) include the current assignment participant and when the current assignment participant was reviewed by the current reviewer (ap) it should insert the related information and return the corresponding collusion cycle.
  4. When the reviewers of the current reviewer (ap) include the current assignment participant and the assignment participant did not submit a review of the current reviewer (ap) it should skip the current reviewer (ap) and return the corresponding collusion cycle.
  5. When the reviewers of the current reviewer (ap) include the current assignment participant and the assignment participant did submit a review of the current reviewer (ap) it should skip the current reviewer (ap) and return the corresponding collusion cycle.
• Test Three-Node Cycle
  1. When the reviewers of the current reviewer (ap2) do not include the current assignment participant it should skip the current reviewer (ap2) and return the corresponding collusion cycle.
  2. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current assignment participant did not receive a review from the current reviewee (ap1) it should skip the current reviewer (ap2) and return the corresponding collusion cycle.
  3. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current assignment participant received a review from the current reviewee (ap1) it should insert the related information and return the corresponding collusion cycle.
  4. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current reviewee (ap1) received a review from the current reviewer (ap2) it should insert the related information and return the corresponding collusion cycle.
  5. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current reviewer (ap2) received a review from the current assignment participant it should insert the related information and return the corresponding collusion cycle.
  6. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current reviewer (ap2) did not receive a review from the current assignment participant it should skip the current reviewer (ap2) and return the corresponding collusion cycle.
  7. When the reviewers of the current reviewer (ap2) include the current assignment participant and the current reviewee (ap1) did not receive a review from the current reviewer (ap2) it should skip the current reviewer (ap2) and return the corresponding collusion cycle.

• Test Four-Node Cycle
  1. When the reviewers of the current reviewer (ap3) do not include current assignment participant it should skip the current reviewer (ap3) and return the corresponding collusion cycle.
  2. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current assignment participant received a review from the reviewee of the current reviewee (ap1) it should insert the related information into the collusion cycle and return results.
  3. When the reviewers of the current reviewer (ap3) include the current assignment participant and the reviewee of the current reviewee (ap1) received a review from the current reviewee (ap2) it should insert the related information into the collusion cycle and return results.
  4. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current reviewee (ap2) received a review from the current reviewer (ap3) it should insert the related information into the collusion cycle and return the results.
  5. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current reviewer (ap3) received a review from the current assignment participant it should insert the related information into the collusion cycle and return the results.
  6. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current assignment participant did not receive a review from the reviewee of the current reviewee (ap1) it should skip the current reviewer (ap3) and return the corresponding collusion cycle.
  7. When the reviewers of the current reviewer (ap3) include the current assignment participant and the reviewee of the current reviewee (ap1) did not receive a review from the current reviewee (ap2) it should skip the current reviewer (ap3) and return the corresponding collusion cycle.
  8. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current reviewee (ap2) did not receive a review from the current reviewer (ap3) it should skip the current reviewer (ap3) and return the corresponding collusion cycle.
  9. When the reviewers of the current reviewer (ap3) include the current assignment participant and the current reviewer (ap3) did not receive a review from the current assignment participant it should skip the current reviewer (ap3) and return the corresponding collusion cycle.

• Test Cycle Similarity Score
  1. When the collusion cycle has been calculated it should return a similarity score that is equal to the average difference between all of the scores in a two-node cycle.
  2. When the collusion cycle has been calculated it should return a similarity score that is equal to the average difference between all of the scores in a three-node cycle.
  3. When the collusion cycle has been calculated it should return a similarity score that is equal to the average difference between all of the scores in a four-node cycle.
• Test Cycle Deviation Score
  1. When the collusion cycle has been calculated it should return a deviation score based on the appropriate values in a two-node cycle.
  2. When the collusion cycle has been calculated it should return a deviation score based on the appropriate values in a three-node cycle.
  3. When the collusion cycle has been calculated it should return a deviation score based on the appropriate values in a four-node cycle.

Description of the Test

Cycle Tests

Each of the tests are very similar in the way they are tested. With the addition of each node in the cycle, the underlying process remains the same. This means that only the number of branches increases, which is the only reason there are more tests in the Four-Node Cycle section then there are in the Two-Node Cycle section. To test the cycles we used stubs to create the relationships between each of the assignment participants in a before-each block before each cycle tests. Then we applied the changes specified to test each branch. To break up the code more easily we separated the testing of each branch into a context block that provided additional information about the test itself as well as making the code look much neater.

Similarity and Deviation Tests

The Similarity and Deviation tests use calculations to determine the similarity and the deviation of each of the scores. We can verify these results by creating a mock data set of reviews and participants and calculating the scores by hand then checking to make sure the collusion cycle methods got the same answer. Since these sections only involved calculations there were not many various branches to test other than making sure that scoring methods worked with two, three, and four node cycles.

Notes on test case documentation

• When a test case states that it inserts the 'related information' into the collusion cycle it means that it inserts the review score as well as the appropriate participant into a two-dimensional array.

Design and Impact

Design Choices

Since we were tasked to write test cases rather than programming design cannot be done in the same way one would design a system. Instead, our design revolved more around the use of RSpec syntax to allow for good self-documentation. In addition to self-documentation, we also wanted an easy way to translate the test write-ups that cover every possible branch of the code into actual test code. To do this we made frequent use of the 'context' statement in ruby. The context statement allows us to nest code in the same way that branches are nested while also allowing a direct correlation of test plan writeup to test plan code.

Here is our original skeleton of the test plan

describe '#two_node_cycles' do
  # Branch 1
  context 'basic filler description' do
    it 'description of result' do
      # Test code here
    end
  end

  # Branch 2
  context 'basic filler description 2' do
    it 'description of result' do
      # Test code here
    end
  end
end

Here is the actual snippet of implementation code

describe '#two_node_cycles' do
  # Branch 1
  context 'when current assignment participant was not reviewed by current reviewer (ap)' do
    it 'skips current reviewer (ap) and returns corresponding collusion cycles' do
      allow(participant).to receive(:reviews_by_reviewer).with(participant2).and_return(nil)
      expect(@cycle.two_node_cycles(participant)).to eq([])
    end
  end

  # Branch 2
  context 'when current assignment participant was reviewed by current reviewer (ap)' do
    it 'inserts related information into collusion cycles and returns results' do
      allow(ReviewResponseMap).to receive(:where).with(reviewee_id: team.id, reviewer_id: participant2.id).and_return([response_map_team_1_2])
      allow(Response).to receive(:where).with(map_id: [response_map_team_1_2]).and_return([response_1_2])
      allow(ReviewResponseMap).to receive(:where).with(reviewee_id: team2.id, reviewer_id: participant.id).and_return([response_map_team_2_1])
      allow(Response).to receive(:where).with(map_id: [response_map_team_2_1]).and_return([response_2_1])
      expect(@cycle.two_node_cycles(participant)[0][0]).to eq([participant, 90])
    end
  end
end

Impact Analysis

Currently, we have already found one bug, which was a missing parameter in the collusion_cycle.rb file with each of the node_cycle methods (two_node_cycle, three_node_cycle, four_node_cycle) which needed an assignment participant parameter to be passed in for the function to even work. In addition the obvious benefit of adding more coverage you a better assurance that the code works well. Of course, testing does not guarantee that the code works, but can only be used to find where the code does not work.

Running the Test Files

To run the test files you need to clone the repository in an Expertiza environment Expertiza Github. Instructions for creating an appropriate Expertiza environment can be found in the README file in the Expertiza GitHub repository.

Once you have the Expertiza repository setup make sure you are in the root directory of the app and run `rspec spec/models/collusion_cycle_spec.rb`

For a demonstration and you can check out this video on YouTube.

Future Work

Because the scope of this addition is relatively small, and we have tested all possible branches within the code there is not much future work to do other than possibly refactor the code with a more efficient tool. Test coverage through quality tests consistently added throughout the development of the Expertiza project should be the future work of the project.