CSC/ECE 517 Spring 2023 - E2336. Reimplement response.rb

Project Overview

This project is a continuation to E2312. The current project is to make design changes to the Response class, ReviewCommentMixin module, ScorableMixin module and add comments to test cases. We explain the issue with respect to the current implementation of each of these entities as subsections below.

Note to the reviwer for final submission

• This project is a refactoring of some files done in Project 3. We have made these changes in a separate repo so that the new changes can be seen clearly.

• We have added a new section in this documentation called Changes made / Solutions that has all the changes we have made and explanations.

PR for this project: https://github.com/expertiza/reimplementation-back-end/pull/38

The PR for the previous project (Project 3) on top of which these changes have been made: https://github.com/expertiza/reimplementation-back-end/pull/23

• Check the beginning of the test plan section on how to test the functionality.

Issues

There are several issues we plan to address as part of this project.

• Refactor the get_all_review_comments method as an instance method in response.rb file
• Refactoring significant_difference
• Make the method in ReviewCommentMixin to be an instance method
• Simplify + Refactor methods in ScorableMixin
• Refactor the self.get_all_responses method as an instance method in response_map.rb file.
• Add more descriptive comments for the tests.

Issue 1: Refactor the self.get_all_review_comments method

Currently, the get_all_review_comments method is a class method that returns a data structure containing all the review comments for all responses. This violates the principles of good design as it creates a large data structure that the calling method needs to break apart, leading to poor performance and increased complexity.

To address this issue and come up with a much more elegant design, we plan to refactor as a new implementation that should convert the get_all_review_comments method into an instance method that takes a review object as an argument and returns the comments for that review only. This will allow the client class to use an Iterator pattern to fetch one review at a time from the Response class rather than iterate over that data structure that get_all_review_comments returned to it.

Issue 2: Refactoring significant_difference?

While the significant_difference method does not necessarily need to be refactored, it is the only method with a call for the avg_scores_and_count_for_prev_reviews method, which does need to be refactored it both is a class method and returns a data structure.

 def self.avg_scores_and_count_for_prev_reviews(existing_responses, current_response)
   scores_assigned = []
   count = 0
   existing_responses.each do |existing_response|
     unless existing_response.id == current_response.id # the current_response is also in existing_responses array
       count += 1
       scores_assigned << existing_response.aggregate_questionnaire_score.to_f / existing_response.maximum_score
     end
   end
   [scores_assigned.sum / scores_assigned.size.to_f, count]
 end

Because of this we decided that we should simply refactor significant_difference instead and simply remove avg_scores_and_count_for_prev_reviews.

 # compare the current response score with other scores on the same artifact, and test if the difference
 # is significant enough to notify instructor.
 # Precondition: the response object is associated with a ReviewResponseMap
 ### "map_class.assessments_for" method need to be refactored
 def significant_difference?
   map_class = map.class
   existing_responses = map_class.assessments_for(map.reviewee)
   average_score_on_same_artifact_from_others, count = Response.avg_scores_and_count_for_prev_reviews(existing_responses, self)
   # if this response is the first on this artifact, there's no grade conflict
   return false if count.zero?
   # This score has already skipped the unfilled scorable question(s)
   score = aggregate_questionnaire_score.to_f / maximum_score
   questionnaire = questionnaire_by_answer(scores.first)
   assignment = map.assignment
   assignment_questionnaire = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: questionnaire.id)
   # notification_limit can be specified on 'Rubrics' tab on assignment edit page.
   allowed_difference_percentage = assignment_questionnaire.notification_limit.to_f
   # the range of average_score_on_same_artifact_from_others and score is [0,1]
   # the range of allowed_difference_percentage is [0, 100]
   (average_score_on_same_artifact_from_others - score).abs * 100 > allowed_difference_percentage
 end

Issue 3: Make the method in ReviewCommentMixin to be an instance method

Problems

• The ReviewCommentMixin has one single method volume_of_review_comments and it is currently present as a class method. There is no need for it to be a class method and it should instead be an instance method.
• Volume of review comments belongs in assignment.rb, not in a mixin for responses.

Proposed Solution

The change is quite straight forward. We should move this method to be an instance method in assignment.rb file. A side note is that, this method calls the get_all_review_comments method from response.rb. This method will be changed as part of this project (See above points). Any change to that function's method call must be reflected here too.

Issue 4: Simplify + Refactor methods in ScorableMixin

This mixin has methods that are related to scoring. https://expertiza.csc.ncsu.edu/index.php/Scoring_%26_Grading_Methods_(Fall_%2721) is a good resource to understand how scoring and grading works.

Problems

• 1) Not DRY : The following piece of code is used in 2 functions - calculate_total_score and maximum_score

 scores.each do |s|
   question = Question.find(s.question_id)
   < Perform a calculation >
 end

Basically, we iterate through the score and calculate something. The common calculation between both these functions is total_weight calculation.

What problem does this cause? If the logic to calculate the total_weight changes in the future, we will have to make changes to both these places. A developer who is relatively new to the project might miss out one changing it in a location. It is always good to have a single point of change by 'DRY'ing our code.

Update for final submission: We have decided not to do this since the repeated code was only of 2 lines and that is too small to be considered not DRY.

• 2) Not optimized: Another change related to the above snippet has to do with how the queries are made and the number of queries.

 scores.each do |s|
   question = Question.find(s.question_id)
   <Other lines irrelevant to this case >
 end

What problem does this cause? This is done inside a loop, meaning we perform a select query to each time -> N queries for N objects. Our solution aims to optimize this.

• 3) Checking the class type of an object to perform some business logic

This issue is present in the questionnaire_by_answer method. The following snippet shows it:

 assignment = if map.is_a? ReviewResponseMap
   map.assignment
 else
   Participant.find(map.reviewer_id).assignment
 end

Proposed Solution

• 1) The common function would involve calculating the sum of weights.

 def calculate_total_weight()
   scores.each do |s|
   question = Question.find(s.question_id)
   total_weight += question.weight unless s.answer.nil? || !question.is_a?(ScoredQuestion)
 end
 And this can be called inside the calculate_total_score and maximum_score.

Update for final submission: We have decided not to do this since the repeated code was only of 2 lines and that is too small to be considered not DRY.

• 2) Instead of calling Question.find inside the loop, we can get the set of question_ids and use a where function.

 question_ids = scores.map(&:question_id)
 questions = Question.where(question_id: question_ids)
 < Make the calculation here >
 This way, we only make one database call instead of N calls.

• 3) To solve checking the object type, we can write a method named response_assignment in ResponseMap and ReviewResponseMap. This way, we can just call that method in the object and depending on what object it is, we would get the corresponding answer.

  * In ReviewResponseMap, the method should return its assignment
  * In ResponseMap, the method should return Participant.find(map.reviewer_id).assignment

A visual representation of this change would look like this. The snippet uses an 'OR' condition to pick the appropriate assignment. Our change would remove the need for this check.

Issue 5: Refactor the self.get_all_responses method

The self.get_all_responses method which is currently in the response.rb file would be much cleaner as an instance method in response_map.rb. We plan to refactor the self.get_all_responses method in response.rb to an instance method in response_map.rb.

Issue 6: Add more descriptive comments for the tests

The existing tests need more descriptive comments. We plan to add more detailed comments for all the test cases.

Changes made / Solutions

Issue 1

The get_all_review_comments method has been moved to be an instance method of the assignment class and modified to only take the reviewee_id. However, the return value has not been modified. This is because upon further investigation the get_all_review_comments (previously called concatenate_all_review_comments) is only used within the volume_of_review_comments method and that modifying volume_of_review_comments to only get a single variable from get_all_review_comments proved to difficult. Unlike the avg_scores_and_count_for_prev_reviews, however, get_all_review_comments was not merged into get_all_review_comments due to its size and that doing so would make get_all_review_comments difficult to understand.

Issue 2

The avg_scores_and_count_for_prev_reviews method has been merged with the significant_difference? method.

 def significant_difference?
   map_class = map.class
   # gets all responses made by a reviewee
   existing_responses = map_class.assessments_for(map.reviewee)
   #
   count = 0
   total = 0
   # gets the sum total percentage scores of all responses that are not this response
   existing_responses.each do |response|
     unless id == response.id # the current_response is also in existing_responses array
       count += 1
       total +=  response.aggregate_questionnaire_score.to_f / response.maximum_score
     end
   end
   #
   # if this response is the only response by the reviewee, there's no grade conflict
   return false if count.zero?
   #
   # calculates the average score of all other responses
   average_score = total / count
   #
   # This score has already skipped the unfilled scorable question(s)
   score = aggregate_questionnaire_score.to_f / maximum_score
   questionnaire = questionnaire_by_answer(scores.first)
   assignment = map.assignment
   assignment_questionnaire = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: questionnaire.id)
   #
   # notification_limit can be specified on 'Rubrics' tab on assignment edit page.
   allowed_difference_percentage = assignment_questionnaire.notification_limit.to_f
   #
   # the range of average_score_on_same_artifact_from_others and score is [0,1]
   # the range of allowed_difference_percentage is [0, 100]
   (average_score - score).abs * 100 > allowed_difference_percentage
 end

Issue 3

We have made the volume_of_review_comments method to be an instance method and moved it to the assignment.rb since it used to take a 'assignment_id' as an input argument. In this way, we are able to make it more sense and removed the ReviewCommentMixin.

   def volume_of_review_comments(assignment_id, reviewer_id)
     comments, counter,
       @comments_in_round, @counter_in_round = Response.get_all_review_comments(assignment_id, reviewer_id)
     # Index 0 is a nil element that can be ignored in the round count
     num_rounds = @comments_in_round.count - 1
 
     overall_avg_vol = (Lingua::EN::Readability.new(comments).num_words / (counter.zero? ? 1 : counter)).round(0)
     review_comments_volume = []
     review_comments_volume.push(overall_avg_vol)
     (1..num_rounds).each do |round|
       num = Lingua::EN::Readability.new(@comments_in_round[round]).num_words
       den = (@counter_in_round[round].zero? ? 1 : @counter_in_round[round])
       avg_vol_in_round = (num / den).round(0)
       review_comments_volume.push(avg_vol_in_round)
     end
     review_comments_volume
   end

Issue 4

• 1) We have decided not to address this part of the issue (See the Issue 4) after discussing with the professor. The reason behind it is that the lines involved in this section is only 2 lines long and hence it does not have to be moved to a method.

• 2) We looked at a lot of solutions to make this. The issue is that using just a 'where' method will not ensure that the Active Record Association returned will be in the same order of that of the list of ids given.

For example:

 ids = [4,1,2,7]
 answers = Answer.where(id: ids)
 return answers.map(&:id)

This example snippet would return 1,2,4,7

Hence, to ensure the order of the return values(thereby ensuring the correct working) and to improve the performance, we have used a small gem 'find_with_order'. The main reason why we decided to add this gem over a custom solution is portability / ease of modification. A custom solution would have required us to write SQL queries in where clauses and this would be tied to the mysql(current database used). This gem is compatible with popular databases and hence this would be automatically taken care of even if the database is changed.

Hence, our final solution is:

   def calculate_total_score
     # Only count the scorable questions, only when the answer is not nil (we accept nil as
     # answer for scorable questions, and they will not be counted towards the total score)
     sum = 0
     question_ids = scores.map(&:question_id)
     # We use find with order here to ensure that the list of questions we get is in the same order as that of question_ids
     questions = Question.find_with_order(question_ids)
     scores.each_with_index do |score, idx|
       sum += score.answer * questions[idx].weight unless score.answer.nil? || !questions[idx].is_a?(ScoredQuestion)
     end
     sum
   end

 def maximum_score
   # Only count the scorable questions, only when the answer is not nil (we accept nil as
   # answer for scorable questions, and they will not be counted towards the total score)
   total_weight = 0
   question_ids = scores.map(&:question_id)
   # We use find with order here to ensure that the list of questions we get is in the same order as that of question_ids
   questions = Question.find_with_order(question_ids)
   scores.each_with_index do |score, idx|
     total_weight += questions[idx].weight unless score.answer.nil? || !questions[idx].is_a?(ScoredQuestion)
   end
   questionnaire = if scores.empty?
                     questionnaire_by_answer(nil)
                   else
                     questionnaire_by_answer(scores.first)
                   end
   total_weight * questionnaire.max_question_score
 end

• 3) Usage of visitor pattern to avoid checking class types: We solved this by adding a method response_assignment to ReviewResponseMap and ResponseMap classes.

In response_map.rb

 # returns the assignment related to the response map
 def response_assignment
   return Participant.find(self.reviewer_id).assignment
 end

In review_response_map.rb

 # returns the assignment related to the review response map
 def response_assignment
   return self.assignment
 end

Hence, the questionnaire_by_answer method in ScorableHelper will have the following snippet:

     map = ResponseMap.find(map_id)
     assignment = map.response_assignment

Issue 5

The get_all_responses method has been made an instance method of response_map and simplified as the function no longer need to find a response map.

 def get_all_responses
   Response.where(map_id: map_id).all
 end

Issue 6

Additional comments have been made to the tests for the response class.

Design Principles

This section aims to act as a consolidation of the design principles that we will implement as part of this project

• DRY: We will DRY the code that we handle as part of this project. An example of this would be the change that we make as part of the ScorableMixin (see Issue 4).

• Visitor Pattern: Having to check the class of an object and making calls based on that is an example of a bad design pattern. An example of this is ScorableMixin. We will adopt a loose implementation of the Visitor pattern to overcome this by adding a method in ResponseMap and ReviewResponseMap and just call that method in the object we receive. See Issue 4 section for a detailed explanation.

Test Plan

Our is a refactor project for models. Hence, to test the functionality, we have ensured to cover existing test cases and add new test cases for any new code we have added.

To run rspec for this project,

rspec spec/models/response_spec.rb

As part of this section, we track the set of files that are to be changed for tests and scenarios that are to be tested.

Files

Since this project is tied to the Response class, all the test cases will be added to response_spec.rb

• Issue 1,2,4: Modify tests in response_spec.rb
• Issue 3: Write tests for volume_of_review_comments method

Scenarios

• Test the score calculation cases - maximum possible total score for all scores, average score across all of the instances and total score (weighted sum). Ideally average score calculation should cover all the scenarios, that is, if the maximum score is zero, it uses the total score(weighted).
• Test comments for review - We also want to test the scenario that involves seeing the comments for a review.

Future Work

The issues pointed out in this documentation will be implemented once the design gets approved. Further changes will be updated in this section as we make progress with the project.

Mentors

• Prof. Edward F. Gehringer

Contributors

• Arvind Srinivas Subramanian (asubram9)
• David Brain (mdbrain2)
• Dhanya Sri Dasari (ddasari)
• Zhihao Wang (zwang238@ncsu.edu)