CSC/ECE 517 Spring 2016/Refactor review mapping controller.rb: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(30 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==E1615. Refactoring the Review Mapping Controller==
This page provides details about the OSS project which was based on refactoring one of controllers related to peer reviewing strategies used in Expertiza.


This page provides details about the OSS project which was based on refactoring one of controllers related to peer reviewing strategies used in Expertiza.


=='''Expertiza '''==
The Expertiza is a software project to create reusable learning objects through peer review. It is a Ruby on Rails based application which can be run on Windows, Linux and Mac OS X.


__TOC__
Main Features of Expertiza are:
* Allows students to work in groups to improve others learning experiences
* The work done by students/team is subjeted to multiple reviews , minimizing plagiarism
* Reviewing is done by studnets, which allows instructor/TA to spend less time in grading
* Large classes can also be handled easily with the help of Expertiza


=='''About Review mapping controller'''==


===Introduction to Expertiza===
Review mapping controller contains methods related to peer reviewing strategies. It contains methods to add a reviewer, delete a reviewer, selecting a reviewer. Depending on the number of students and number of submissions, the topics to be reviewed are assigned to the students automatically. If a user wants to look for the team for a submission , it returns the team by comparing the submission id's with the team id's. Also, it assigns quizzes dynamically. Generation of review report, feedback report and teammate review is done.


[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.
==='''Problem Statement'''===


===Problem Statement===
The main aim of this project is to
The following tasks were accomplished in this project:


* Improved the clarity of code by improving the variable and parameter names.
* Refactor reporting method(response_report) which is big
* Long character strings were taken and given appropriate names.
* Use more efficient function sample instead of shuffle for random number selection
* Handled pagination by a separate helper module, which can be used by multiple controllers.
* Remove unused and assigned variables
* Implemented action_allowed for access_control  to prevent unauthorized access of methods.
* Simplify Automatic_review_mapping_strategy
* Prevented displaying of all versions for all users and tables when a user views the index page.
* Added missing CRUD methods to Versions Controller
* Added RSPEC testcases for testing changes done in Versions Controller


===About Versions Controller===
=='''Code Improvements'''==
This class manages different versions of reviews.  If a reviewer reviews a submission, and after that, the author revises the submission, the next time the reviewer does a review (s)he will create a new version.  Sometimes it’s necessary to find the current version of a review; sometimes it’s necessary to find all versions.  Similarly, a user may want to delete the current version of a review, or all versions of a review.
Pagination of versions helps the user to view a subset of versions at a time. Considering the huge number of versions in the system, it is very useful to have a pagination mechanism and a filtering mechanism which can be applied on the whole set of versions. The idea is to display the versions in an ordered, comprehensible and logical manner. In Expertiza the gem ‘will_paginate’ is used to achieve pagination.


===Current Implementation===
==='''Unused variables and arguments'''===


There are unused variables in the methods which use the stack unnecessarily. So, it is better to remove the unused variables or at the least indicate that a variable is unused.


=====Functionality=====
For suppose when both keys and values are not used in a hash but are given as arguments, then the unused variables can be indicated by adding a "_" infront of the name or replace the unused variable with "_" to represent it as unused variable but allow them in arguments.
* Any user irrespective of his/ her privileges can view all the versions.
::The versions which a particular user can view should be restricted based on the privileges of the user. For instance, only a user with Administrator privileges should be able to view all the versions in the system. However, that is not the case now. Every user can view all the versions irrespective of whether the user is a student or an administrator.
* Any user can delete any version
::The versions which a particular user can delete should be restricted based on the privileges of the user. For instance, a student should not be allowed to delete any version. According to the current implementation any user can delete any version in the system.
* Filtering of versions were restricted to the current user
::The filtering options on versions were restricted to the current user. Sometimes a user might want to view versions associated with other users. For instance, an instructor might want to view the list of versions created by a particular student. This is not possible with the current implementation.


=====Drawbacks and Solutions=====
* '''Problem 1''': The method paginate_list is doing more than one thing.
::The method paginate_list was building a complex search criteria based on the input params, getting the list of versions from the Database matching this search criteria and then calling the Page API. All these tasks in a single method made it difficult to understand.
<pre>
<pre>
# For filtering the versions list with proper search and pagination.
Previous Code: teams_hash = unsorted_teams_hash.sort_by{|k, v| v}.to_h
  def paginate_list(id, user_id, item_type, event, datetime)
</pre>
    # Set up the search criteria
<pre>
    criteria = ''
After Changing the code: teams_hash = unsorted_teams_hash.sort_by{|_, v| v}.to_h
    criteria = criteria + "id = #{id} AND " if id && id.to_i > 0
</pre>
    if current_user_role? == 'Super-Administrator'
      criteria = criteria + "whodunnit = #{user_id} AND " if user_id && user_id.to_i > 0
    end
    criteria = criteria + "whodunnit = #{current_user.try(:id)} AND " if current_user.try(:id) && current_user.try(:id).to_i > 0
    criteria = criteria + "item_type = '#{item_type}' AND " if item_type && !(item_type.eql? 'Any')
    criteria = criteria + "event = '#{event}' AND " if event && !(event.eql? 'Any')
    criteria = criteria + "created_at >= '#{time_to_string(params[:start_time])}' AND "
    criteria = criteria + "created_at <= '#{time_to_string(params[:end_time])}' AND "


    if current_role == 'Instructor' || current_role == 'Administrator'
In the above case teams_hash should consist of a hash with both keys and values but the sorting is done based on values. So the key is replaced with a "_" so that the user may deem it unused in the implementation of the process.


    end
==='''Use sample instead of shuffle'''===


    # Remove the last ' AND '
When sample is used, the elements in an array are chosen by using random and unique indices in the array so that the elements doesn't repeat in the array. This cannot be guaranteed in shuffle. Also by using shuffle[0] we are shuffling all the elements in the array and then picking the first element instead of picking a single element randomly which is more efficient. The following are the couple of places where shuffle[0] was used and is replaced by sample.
    criteria = criteria[0..-5]


    versions = Version.page(params[:page]).order('id').per_page(25).where(criteria)
<pre>
    versions
Previous Code:
  end
</pre>
* '''Solution''': The implementation has been changed in such a way that the versions which a user is allowed to see depends on the privileges of the user. The approach we have taken is as follows:
**An administrator can see all the versions
**An instructor can see all the versions created by him and other users who are in his course or are participants in the assignments he creates.
**A TA can see all the versions created by him and other users who are in the course for which he/ she assists.
**A Student can see all the versions created by him/ her.
* '''Problem 2''': The search criteria created in the method paginate_list was difficult to comprehend.
::The code which builds the search criteria in the method paginate_list uses many string literals and conditions and is hardly intuitive. The programmer will have to spend some time to understand what the code is really doing.
* '''Solution''': The implementation has been changed. A student is not allowed to delete any versions now. Other types of users, for instance administrators, instructors and TAs are allowed to delete only the versions they are authorized to view.
* '''Problem 3''': The paginate method can be moved to a helper class.
::VersionsController is not the only component which require to paginate items. There are other components too. For instance, the UsersController has to paginate the list of users. Hence the Paginate method can be moved to a helper class which can be accessed by other components as well.
* '''Solution''': The filtering options has also been enhanced. The current user can now choose as part of the version search filter any user from a list of users if the current user is authorized to see the versions created by that user.


===New Implementation===
assignment_team = assignment_teams.to_a.shuffle[0] rescue nil
*The method paginate_list has been split into 2 methods now.
** BuildSearchCriteria – as the name suggests the sole purpose of this method is to build a search criteria based on the input search filters when the current user initiates a search in versions.
** paginate_list – this method will call the paginate API.
:First the search criteria is built, then the criteria is applied to versions in the database to get all versions which matches the criteria and then the retrieved versions are paginated.
<pre>
  # pagination.
  def paginate_list(versions)
    paginate(versions, VERSIONS_PER_PAGE);
  end


  def BuildSearchCriteria(id, user_id, item_type, event)
topic = assignment.candidate_topics_to_review(reviewer).to_a.shuffle[0] rescue nil
    # Set up the search criteria
    search_criteria = ''
    search_criteria = search_criteria + add_id_filter_if_valid(id).to_s
    if current_user_role? == 'Super-Administrator'
      search_criteria = search_criteria + add_user_filter_for_super_admin(user_id).to_s
    end
    search_criteria = search_criteria + add_user_filter
    search_criteria = search_criteria + add_version_type_filter(item_type).to_s
    search_criteria = search_criteria + add_event_filter(event).to_s
    search_criteria = search_criteria + add_date_time_filter
    search_criteria
  end
</pre>
</pre>
* The string literals and conditions in the method paginate_list were replaced with methods with intuitive names so that the programmer can understand the code more easily. We also removed an empty if clause and a redundant statement.
<pre>
<pre>
  def add_id_filter_if_valid (id)
After Changing the code:
    "id = #{id} AND " if id && id.to_i > 0
 
  end
assignment_team = assignment_teams.to_a.sample rescue nil


  def add_user_filter_for_super_admin (user_id)
topic = assignment.candidate_topics_to_review(reviewer).to_a.sample rescue nil
    "whodunnit = #{user_id} AND " if user_id && user_id.to_i > 0
</pre>
  end


  def add_user_filter
==='''Cyclomatic complexity of automatic_review_mapping_strategy method'''===
    "whodunnit = #{current_user.try(:id)} AND " if current_user.try(:id) && current_user.try(:id).to_i > 0
  end


  def add_event_filter (event)
The method automatic_review_mapping_strategy handles the number of reviews assigned to a individual and also the number of reviews that can be done with in a team. The method is very long and has many nested if statements due to which the complexity of the method is very high. Instead of a single method handling all the parts of the strategy, it is divided into several parts due to which the code is more readable and also the complexity of code is shared by each method.
    "event = '#{event}' AND " if event && !(event.eql? 'Any')
  end


  def add_date_time_filter
The method first checks the number of participants that are in an assignment and the number of teams that are present. It then sets the values for the maximum number of reviews that are possible and also the minimum number that are required. Then it assigns the reviews to each of the teams randomly when a request for review is made by a participant. At last it checks if the participant has the minimum number of reviews required after allotting the reviews. If not, it assigns more reviews to valid teams so that the minimum requirement is met.
    "created_at >= '#{time_to_string(params[:start_time])}' AND " +
        "created_at <= '#{time_to_string(params[:end_time])}'"
  end


  def add_version_type_filter (version_type)
The part of the code that is moved out of automatic_review_mapping_strategy as peer_review_strategy:
    "item_type = '#{version_type}' AND " if version_type && !(version_type.eql? 'Any')
  end
</pre>
* The paginate method has been moved to the helper class Pagination_Helper. This new method can be now reused by the different components like UsersController etc. The method receives two parameters, first the list to paginate and second the number of items to be displayed in a page.


<pre>
<pre>
module PaginationHelper
private
def peer_review_strategy(teams, num_participants, student_review_num, participants, participants_hash)
iterator = 0
teams.each do |team|
  selected_participants = Array.new
  if !team.equal? teams.last
#need to even out the # of reviews for teams
while selected_participants.size < num_reviews_per_team
  num_participants_this_team = TeamsUser.where(team_id: team.id).size
  #If there are some submitters or reviewers in this team, they are not treated as normal participants.
  #They should be removed from 'num_participants_this_team'
  TeamsUser.where(team_id: team.id).each do |team_user|
temp_participant = Participant.where(user_id: team_user.user_id, parent_id: assignment_id).first
num_participants_this_team -= 1 if temp_participant.can_review == false or temp_participant.can_submit == false
  end
  #if all outstanding participants are already in selected_participants, just break the loop.
  break if selected_participants.size == participants.size - num_participants_this_team
 
  # generate random number
  if iterator == 0
rand_num = rand(0..num_participants-1)
  else
min_value = participants_hash.values.min
#get the temp array including indices of participants, each participant has minimum review number in hash table.
participants_with_min_assigned_reviews = Array.new
participants.each do |participant|
  participants_with_min_assigned_reviews << participants.index(participant) if participants_hash[participant.id] == min_value
end
#if participants_with_min_assigned_reviews is blank
if_condition_1 = participants_with_min_assigned_reviews.empty?
#or only one element in participants_with_min_assigned_reviews, prohibit one student to review his/her own artifact
if_condition_2 = (participants_with_min_assigned_reviews.size == 1 and TeamsUser.exists?(team_id: team.id, user_id: participants[participants_with_min_assigned_reviews[0]].user_id))
if if_condition_1 or if_condition_2
  #use original method to get random number
  rand_num = rand(0..num_participants-1)
else
  #rand_num should be the position of this participant in original array
  rand_num = participants_with_min_assigned_reviews[rand(0..participants_with_min_assigned_reviews.size-1)]
end
  end
  # prohibit one student to review his/her own artifact
  next if TeamsUser.exists?(team_id: team.id, user_id: participants[rand_num].user_id)


  def paginate (items, number_of_items_per_page)
  if_condition_1 = (participants_hash[participants[rand_num].id] < student_review_num)
    items.page(params[:page]).per_page(number_of_items_per_page)
  if_condition_2 = (!selected_participants.include? participants[rand_num].id)
  end
  if if_condition_1 and if_condition_2
# selected_participants cannot include duplicate num
selected_participants << participants[rand_num].id
participants_hash[participants[rand_num].id] += 1
  end
  # remove students who have already been assigned enough num of reviews out of participants array
  participants.each do |participant|
if participants_hash[participant.id] == student_review_num
  participants.delete_at(rand_num)
  num_participants -= 1
end
  end
end
  else
#review num for last team can be different from other teams.
#prohibit one student to review his/her own artifact and selected_participants cannot include duplicate num
participants.each do |participant|
  # avoid last team receives too many peer reviews
  if !TeamsUser.exists?(team_id: team.id, user_id: participant.user_id) and selected_participants.size < num_reviews_per_team
selected_participants << participant.id
participants_hash[participant.id] += 1
  end
end
  end


end
  begin
selected_participants.each {|index| ReviewResponseMap.where(:reviewee_id => team.id, :reviewer_id => index, :reviewed_object_id => assignment_id).first_or_create}
  rescue
flash[:error] = "Automatic assignment of reviewer failed."
  end
  iterator += 1
end
end
</pre>
</pre>


===Code improvements===
The method is made private so that it can only be called with in the controller and cannot directly be called through a view.
* Introduced a constant VERSIONS_PER_PAGE and assigned the value 25 to it. The pagination algorithm for VersionsController displays at most 25 versions in a page. The existing implementation uses the value 25 straight in the code and there are few problems associated with such an approach.
 
** It is not easy to understand what 25 is unless the programmer takes a close look at the code.
The complexity of the original method reduced after breaking it and can be easily readable now.
** In case if the value 25 is used at more than one places and in future a new requirement comes to show at most 30 versions in a page, all the values will have to be modified. It is not very DRY.
* The VersionsController was overriding AccessHelper - action_allowed? method to return true in all the cases. This was violating the whole purpose of the method action_allowed?. The purpose of this method is to determine whether the user who is triggering a CRUD operation is allowed to do so. So when the current user invokes a CRUD operation, the action_allowed? method is invoked first and if the method returns true the CRUD operation is triggered or else the user is intimated with a message and gracefully exited. Hence, when the action_allowed? method is overridden to return true always, it results in providing unauthorized access to certain users.


<pre>
<pre>
def action_allowed?
def automatic_review_mapping_strategy(assignment_id, participants, teams, student_review_num=0, submission_review_num=0)
     true
    participants_hash = {}
    participants.each { |participant| participants_hash[participant.id] = 0 }
    #calculate reviewers for each team
    num_participants = participants.size
    if student_review_num != 0 and submission_review_num == 0
      num_reviews_per_team = (participants.size * student_review_num * 1.0 / teams.size).round
      exact_num_of_review_needed = participants.size * student_review_num
    elsif student_review_num == 0 and submission_review_num != 0
      num_reviews_per_team = submission_review_num
      student_review_num = (teams.size * submission_review_num * 1.0 / participants.size).round
      exact_num_of_review_needed = teams.size * submission_review_num
    end
    #Exception detection: If instructor want to assign too many reviews done by each student, there will be an error msg.
    if student_review_num >= teams.size
      flash[:error] = 'You cannot set the number of reviews done by each student to be greater than or equal to total number of teams [or "participants" if it is an individual assignment].'
    end
 
    peer_review_strategy(teams, num_participants, student_review_num, participants, participants_hash)
 
    # after assigning peer reviews for each team, if there are still some peer reviewers not obtain enough peer review, just assign them to valid teams
    if ReviewResponseMap.where(["reviewed_object_id = ? and created_at > ? and calibrate_to = ?", assignment_id, @@time_create_last_review_mapping_record, 0]).size < exact_num_of_review_needed
      participants_with_insufficient_review_num = Array.new
      participants_hash.each do |participant_id, review_num|
        participants_with_insufficient_review_num << participant_id if review_num < student_review_num
      end
      unsorted_teams_hash = {}
      ReviewResponseMap.where(["reviewed_object_id = ? and calibrate_to = ?", assignment_id, 0]).each do |response_map|
        unless unsorted_teams_hash.has_key? (response_map.reviewee_id)
          unsorted_teams_hash[response_map.reviewee_id] = 1
        else
          unsorted_teams_hash[response_map.reviewee_id] += 1
        end
      end
      teams_hash = unsorted_teams_hash.sort_by{|_, v| v}.to_h
      participants_with_insufficient_review_num.each do |participant_id|
        teams_hash.each do |team_id, num_review_received|
          unless TeamsUser.exists?(team_id: team_id, user_id: Participant.find(participant_id).user_id)
            ReviewResponseMap.where(:reviewee_id => team_id, :reviewer_id => participant_id, :reviewed_object_id => assignment_id).first_or_create
            teams_hash[team_id] += 1
            teams_hash = teams_hash.sort_by{|_, v| v}.to_h
            break
          end
        end
      end
    end
     @@time_create_last_review_mapping_record = ReviewResponseMap.where(reviewed_object_id: assignment_id).last.created_at
   end
   end
</pre>
</pre>


:With the new implementation the AccessHelper - action_allowed? method has been modified in such a way that unauthorized access is prevented. As per the new algorithm, 'new', 'create', 'edit', 'update' cannot be invoked by any user. These operations can be accessed only by ‘papertrail’ gem. Only an ‘Administrator’ or ‘Super-Administrator’ can call 'destroy_all' method. All the other methods are accessible to ‘Administrator’,  ‘Super-Administrator’, ‘Instructor’, ‘Teaching Assistant’ and ‘Student’.
'''4. Moving code specific to models to models instead of controller'''
 
The method response_report contains code which generates reports based on the input of one of the three of 'ReviewResponseMap', 'FeedbackResponseMap', 'TeammateReviewResponseMap' and 'Calibration'. Calibration is a special case and should be handled in the controller itself. But the other three are models that are subclasses of ResponseMap model. The report is generated for each of the three by calling the ResponseMap model and obtaining the values by querying the database. Moving the code to the models and calling the methods in the model through the controller makes more sense than writing the code in controller.
 
The following is the code snippet from the original method which contains the calls to ResponseMap model:


<pre>
<pre>
  def action_allowed?
case @type
     case params[:action]
     when "ReviewResponseMap"
    when 'new', 'create', 'edit', 'update'
      if params[:user].nil?
    #Modifications can only be done by papertrail
        # This is not a search, so find all reviewers for this assignment
       return false
        response_maps_with_distinct_participant_id = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id = ? and type = ? and calibrate_to = ?", @id, @type, 0])
     when 'destroy_all'
        @reviewers = []
       ['Super-Administrator',
        response_maps_with_distinct_participant_id.each do |reviewer_id_from_response_map|
      'Administrator'].include? current_role_name
          @reviewers << (AssignmentParticipant.find(reviewer_id_from_response_map.reviewer_id))
     else
        end
       #Allow all others
        @reviewers = Participant.sort_by_name(@reviewers)
       ['Super-Administrator',
      else
      'Administrator',
        # This is a search, so find reviewers by user's full name
      'Instructor',
        user = User.select("DISTINCT id").where(["fullname LIKE ?", '%'+params[:user][:fullname]+'%'])
      'Teaching Assistant',
        @reviewers = AssignmentParticipant.where(["user_id IN (?) and parent_id = ?", user, @assignment.id])
      'Student'].include? current_role_name
      end
      #  @review_scores[reveiwer_id][reviewee_id] = score for assignments not using vary_rubric_by_rounds feature
      # @review_scores[reviewer_id][round][reviewee_id] = score for assignments using vary_rubric_by_rounds feature
      @review_scores = @assignment.compute_reviews_hash
       @avg_and_ranges= @assignment.compute_avg_and_ranges_hash
     when "FeedbackResponseMap"
      #Example query
      #SELECT distinct reviewer_id FROM response_maps where type = 'FeedbackResponseMap' and
       #reviewed_object_id in (select id from responses where
      #map_id in (select id from response_maps where reviewed_object_id = 722 and type = 'ReviewResponseMap'))
      @review_response_map_ids = ResponseMap.select("id").where(["reviewed_object_id = ? and type = ?", @id, 'ReviewResponseMap'])
      @response_ids = Response.select("id").where(["map_id IN (?)", @review_response_map_ids])
      @reviewers = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id IN (?) and type = ?", @response_ids, @type])
     when "TeammateReviewResponseMap"
       #Example query
       #SELECT distinct reviewer_id FROM response_maps where type = 'TeammateReviewResponseMap' and reviewed_object_id = 711
      @reviewers = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id = ? and type = ?", @id, 'TeammateReviewResponseMap'])
     end
     end
   end
   end
</pre>
</pre>


===Automated Testing using RSPEC===
The same code after moving the methods to their respective models looks as follows:
The current version of expertiza did not have any test for VersionsController. Using the test driven development(TDD) approach, we have added an exhaustive set of RSPEC tests for VersionsController, to test all the modifications we have done to the code of the controller class. The tests use double and stub features of rspec-rails gem, to fake the log in by different users - Administrator, Instructor, Student etc. The tests can be executed "rpec spec" command as shown below.
 
<pre>
<pre>
user-expertiza $rspec spec
case @type
.
      when "ReviewResponseMap"
.
        @review_user= params[:user]
.
        #If review response is required call review_response_report method in review_response_map model
Finished in 5.39 seconds (files took 25.33 seconds to load)
        @reviewers = ReviewResponseMap.review_response_report(@id, @assignment,@type, @review_user)
66 examples, 0 failures
        @review_scores = @assignment.compute_reviews_hash
 
        @avg_and_ranges= @assignment.compute_avg_and_ranges_hash
Randomized with seed 19254
      when "FeedbackResponseMap"
.
        #If review report for feedback is required call feedback_response_report method in feedback_review_response_map model
.
        @reviewers = FeedbackResponseMap.feedback_response_report(@id, @type)
      when "TeammateReviewResponseMap"
        #If review report for teammate is required call teammate_response_report method in teammate_review_response_map model
        @reviewers = TeammateReviewResponseMap.teammate_response_report(@id)
    end
  end
</pre>
</pre>
=='''Testing UI'''==


===Testing from UI===
'''Peer review information:'''
Following are a few testcases with respectto our code changes that can be tried from UI:
* Instructor Login:  
1. To go to versions index page, type in the following url after logging in:
        Username: Instructor6
  http://152.46.16.81:3000/versions
        Password: password
 
* Student login:
2. After logging in as student/instructor or admin : Try accessing the  new, create, edit, update actions. These actions are not allowed to any of the users.
        Username: Student6384
  http://152.46.16.81:3000/versions/new
        Password: password
  This calls the new action. In the current production version of expertiza, it is unhandled and application gives a default 404 page.
 
3. Another feature that can be tested from UI is Pagination. Try searching for a user's versions and see if the results are paginated or not. Search here:
  http://152.46.16.81:3000/versions/search


4. Visit the same URL as step 3, you should see only the students under that instructor in the users dropdown.
'''Steps for testing UI:'''
# Login as an instructor (Using Instructor6 will help you import the participants from other assignments).
# Navigate to "Manage->Assignments".
# Click on "New Public Assignment" for creating a new assignment.
# Create a new assignment by providing assignment name, selecting a course, submission directory (Give any name) and description URL.
# Select "has teams?" and provide the team size. Click on create to create a new assignment.
## After that, click on review strategy and limit the number of reviews per submission.
## Click on "Due dates" and update date for submission and review. Adjust the review date and time in order to test the reviews.
## Click on "save". A new assignment is created and it can be viewed in "Manage->Assignments" section.
# In order to add participants, there are two methods to add students to the assignment.
## Click on "add participants" against the assignment. Enter the user login to add the student. Add atleast 6 participants so that the review mapping can be seen.
## Click on any previous assignment "add participants" and export the students list (I used wiki assignment).
## Click on your assignment "add participants" and import the students using the export file.
# Go back to "Assignments" section and click against "create teams".
## After clicking on "create teams", Click on "create teams" in the directed page.
### Teams can be formed either manually or automatically.
# Login as a student to submit the links for assignment (Valid hyperlink must be provided in the submission link). Add submission for atleast 5 students in order to check the automatic review mapping. (Password for student is password)
# After submitting the links for some students, Log in as an instructor to change the assignment phase to review phase.
## To change the review phase period, Go to assignments section and click on edit. Click on due dates and change the review due date.
# Now, login as a student and go to others work. Click on "Request a submission to review" and check whether a review is assigned automatically. If no assignments are submitted, then this cannot be tested.
# For teammate review report, author feedback report and review report, click against "review report" and all the review reports can be seen by selecting it in the drop down menu.


===References===
==References==


#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[https://github.com/harsha1007/expertiza Forked repository for the project]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://expertiza.ncsu.edu/ Expertiza Main Page]
#[http://bit.ly/myexpertiza  Demo link]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#[https://relishapp.com/rspec Rspec Documentation]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

Latest revision as of 22:10, 1 April 2016

This page provides details about the OSS project which was based on refactoring one of controllers related to peer reviewing strategies used in Expertiza.


Expertiza

The Expertiza is a software project to create reusable learning objects through peer review. It is a Ruby on Rails based application which can be run on Windows, Linux and Mac OS X.

Main Features of Expertiza are:

  • Allows students to work in groups to improve others learning experiences
  • The work done by students/team is subjeted to multiple reviews , minimizing plagiarism
  • Reviewing is done by studnets, which allows instructor/TA to spend less time in grading
  • Large classes can also be handled easily with the help of Expertiza

About Review mapping controller

Review mapping controller contains methods related to peer reviewing strategies. It contains methods to add a reviewer, delete a reviewer, selecting a reviewer. Depending on the number of students and number of submissions, the topics to be reviewed are assigned to the students automatically. If a user wants to look for the team for a submission , it returns the team by comparing the submission id's with the team id's. Also, it assigns quizzes dynamically. Generation of review report, feedback report and teammate review is done.

Problem Statement

The main aim of this project is to

  • Refactor reporting method(response_report) which is big
  • Use more efficient function sample instead of shuffle for random number selection
  • Remove unused and assigned variables
  • Simplify Automatic_review_mapping_strategy

Code Improvements

Unused variables and arguments

There are unused variables in the methods which use the stack unnecessarily. So, it is better to remove the unused variables or at the least indicate that a variable is unused.

For suppose when both keys and values are not used in a hash but are given as arguments, then the unused variables can be indicated by adding a "_" infront of the name or replace the unused variable with "_" to represent it as unused variable but allow them in arguments.

Previous Code: teams_hash = unsorted_teams_hash.sort_by{|k, v| v}.to_h
After Changing the code: teams_hash = unsorted_teams_hash.sort_by{|_, v| v}.to_h

In the above case teams_hash should consist of a hash with both keys and values but the sorting is done based on values. So the key is replaced with a "_" so that the user may deem it unused in the implementation of the process.

Use sample instead of shuffle

When sample is used, the elements in an array are chosen by using random and unique indices in the array so that the elements doesn't repeat in the array. This cannot be guaranteed in shuffle. Also by using shuffle[0] we are shuffling all the elements in the array and then picking the first element instead of picking a single element randomly which is more efficient. The following are the couple of places where shuffle[0] was used and is replaced by sample.

Previous Code:

assignment_team = assignment_teams.to_a.shuffle[0] rescue nil

topic = assignment.candidate_topics_to_review(reviewer).to_a.shuffle[0] rescue nil
After Changing the code:

assignment_team = assignment_teams.to_a.sample rescue nil

topic = assignment.candidate_topics_to_review(reviewer).to_a.sample rescue nil

Cyclomatic complexity of automatic_review_mapping_strategy method

The method automatic_review_mapping_strategy handles the number of reviews assigned to a individual and also the number of reviews that can be done with in a team. The method is very long and has many nested if statements due to which the complexity of the method is very high. Instead of a single method handling all the parts of the strategy, it is divided into several parts due to which the code is more readable and also the complexity of code is shared by each method.

The method first checks the number of participants that are in an assignment and the number of teams that are present. It then sets the values for the maximum number of reviews that are possible and also the minimum number that are required. Then it assigns the reviews to each of the teams randomly when a request for review is made by a participant. At last it checks if the participant has the minimum number of reviews required after allotting the reviews. If not, it assigns more reviews to valid teams so that the minimum requirement is met.

The part of the code that is moved out of automatic_review_mapping_strategy as peer_review_strategy:

private
	def peer_review_strategy(teams, num_participants, student_review_num, participants, participants_hash)
		iterator = 0
		teams.each do |team|
		  selected_participants = Array.new
		  if !team.equal? teams.last
			#need to even out the # of reviews for teams
			while selected_participants.size < num_reviews_per_team
			  num_participants_this_team = TeamsUser.where(team_id: team.id).size
			  #If there are some submitters or reviewers in this team, they are not treated as normal participants.
			  #They should be removed from 'num_participants_this_team'
			  TeamsUser.where(team_id: team.id).each do |team_user|
				temp_participant = Participant.where(user_id: team_user.user_id, parent_id: assignment_id).first
				num_participants_this_team -= 1 if temp_participant.can_review == false or temp_participant.can_submit == false
			  end
			  #if all outstanding participants are already in selected_participants, just break the loop.
			  break if selected_participants.size == participants.size - num_participants_this_team

			  # generate random number
			  if iterator == 0
				rand_num = rand(0..num_participants-1)
			  else
				min_value = participants_hash.values.min
				#get the temp array including indices of participants, each participant has minimum review number in hash table.
				participants_with_min_assigned_reviews = Array.new
				participants.each do |participant|
				  participants_with_min_assigned_reviews << participants.index(participant) if participants_hash[participant.id] == min_value
				end
				#if participants_with_min_assigned_reviews is blank 
				if_condition_1 = participants_with_min_assigned_reviews.empty?
				#or only one element in participants_with_min_assigned_reviews, prohibit one student to review his/her own artifact
				if_condition_2 = (participants_with_min_assigned_reviews.size == 1 and TeamsUser.exists?(team_id: team.id, user_id: participants[participants_with_min_assigned_reviews[0]].user_id))
				if if_condition_1 or if_condition_2
				  #use original method to get random number
				  rand_num = rand(0..num_participants-1)
				else
				  #rand_num should be the position of this participant in original array
				  rand_num = participants_with_min_assigned_reviews[rand(0..participants_with_min_assigned_reviews.size-1)]
				end
			  end
			  # prohibit one student to review his/her own artifact
			  next if TeamsUser.exists?(team_id: team.id, user_id: participants[rand_num].user_id)

			  if_condition_1 = (participants_hash[participants[rand_num].id] < student_review_num)
			  if_condition_2 = (!selected_participants.include? participants[rand_num].id)
			  if if_condition_1 and if_condition_2
				# selected_participants cannot include duplicate num
				selected_participants << participants[rand_num].id
				participants_hash[participants[rand_num].id] += 1
			  end 
			  # remove students who have already been assigned enough num of reviews out of participants array
			  participants.each do |participant|
				if participants_hash[participant.id] == student_review_num
				  participants.delete_at(rand_num)
				  num_participants -= 1
				end
			  end
			end
		  else
			#review num for last team can be different from other teams.
			#prohibit one student to review his/her own artifact and selected_participants cannot include duplicate num
			participants.each do |participant| 
			  # avoid last team receives too many peer reviews
			  if !TeamsUser.exists?(team_id: team.id, user_id: participant.user_id) and selected_participants.size < num_reviews_per_team
				selected_participants << participant.id 
				participants_hash[participant.id] += 1
			  end
			end
		  end

		  begin
			selected_participants.each {|index| ReviewResponseMap.where(:reviewee_id => team.id, :reviewer_id => index, :reviewed_object_id => assignment_id).first_or_create}
		  rescue
			flash[:error] = "Automatic assignment of reviewer failed."
		  end
		  iterator += 1
		end
	end

The method is made private so that it can only be called with in the controller and cannot directly be called through a view.

The complexity of the original method reduced after breaking it and can be easily readable now.

def automatic_review_mapping_strategy(assignment_id, participants, teams, student_review_num=0, submission_review_num=0)
    participants_hash = {}
    participants.each { |participant| participants_hash[participant.id] = 0 }
    #calculate reviewers for each team
    num_participants = participants.size
    if student_review_num != 0 and submission_review_num == 0
      num_reviews_per_team = (participants.size * student_review_num * 1.0 / teams.size).round
      exact_num_of_review_needed = participants.size * student_review_num
    elsif student_review_num == 0 and submission_review_num != 0
      num_reviews_per_team = submission_review_num
      student_review_num = (teams.size * submission_review_num * 1.0 / participants.size).round
      exact_num_of_review_needed = teams.size * submission_review_num
    end
    #Exception detection: If instructor want to assign too many reviews done by each student, there will be an error msg.
    if student_review_num >= teams.size
      flash[:error] = 'You cannot set the number of reviews done by each student to be greater than or equal to total number of teams [or "participants" if it is an individual assignment].'
    end

    peer_review_strategy(teams, num_participants, student_review_num, participants, participants_hash)

    # after assigning peer reviews for each team, if there are still some peer reviewers not obtain enough peer review, just assign them to valid teams
    if ReviewResponseMap.where(["reviewed_object_id = ? and created_at > ? and calibrate_to = ?", assignment_id, @@time_create_last_review_mapping_record, 0]).size < exact_num_of_review_needed
      participants_with_insufficient_review_num = Array.new
      participants_hash.each do |participant_id, review_num|
        participants_with_insufficient_review_num << participant_id if review_num < student_review_num
      end
      unsorted_teams_hash = {}
      ReviewResponseMap.where(["reviewed_object_id = ? and calibrate_to = ?", assignment_id, 0]).each do |response_map|
        unless unsorted_teams_hash.has_key? (response_map.reviewee_id)
          unsorted_teams_hash[response_map.reviewee_id] = 1 
        else
          unsorted_teams_hash[response_map.reviewee_id] += 1
        end
      end 
      teams_hash = unsorted_teams_hash.sort_by{|_, v| v}.to_h
      participants_with_insufficient_review_num.each do |participant_id|
        teams_hash.each do |team_id, num_review_received|
          unless TeamsUser.exists?(team_id: team_id, user_id: Participant.find(participant_id).user_id)
            ReviewResponseMap.where(:reviewee_id => team_id, :reviewer_id => participant_id, :reviewed_object_id => assignment_id).first_or_create
            teams_hash[team_id] += 1
            teams_hash = teams_hash.sort_by{|_, v| v}.to_h
            break
          end
        end
      end
    end
    @@time_create_last_review_mapping_record = ReviewResponseMap.where(reviewed_object_id: assignment_id).last.created_at
  end

4. Moving code specific to models to models instead of controller

The method response_report contains code which generates reports based on the input of one of the three of 'ReviewResponseMap', 'FeedbackResponseMap', 'TeammateReviewResponseMap' and 'Calibration'. Calibration is a special case and should be handled in the controller itself. But the other three are models that are subclasses of ResponseMap model. The report is generated for each of the three by calling the ResponseMap model and obtaining the values by querying the database. Moving the code to the models and calling the methods in the model through the controller makes more sense than writing the code in controller.

The following is the code snippet from the original method which contains the calls to ResponseMap model:

case @type
    when "ReviewResponseMap"
      if params[:user].nil?
        # This is not a search, so find all reviewers for this assignment
        response_maps_with_distinct_participant_id = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id = ? and type = ? and calibrate_to = ?", @id, @type, 0])
        @reviewers = []
        response_maps_with_distinct_participant_id.each do |reviewer_id_from_response_map|
          @reviewers << (AssignmentParticipant.find(reviewer_id_from_response_map.reviewer_id))
        end
        @reviewers = Participant.sort_by_name(@reviewers)
      else
        # This is a search, so find reviewers by user's full name
        user = User.select("DISTINCT id").where(["fullname LIKE ?", '%'+params[:user][:fullname]+'%'])
        @reviewers = AssignmentParticipant.where(["user_id IN (?) and parent_id = ?", user, @assignment.id])
      end
      #  @review_scores[reveiwer_id][reviewee_id] = score for assignments not using vary_rubric_by_rounds feature
      # @review_scores[reviewer_id][round][reviewee_id] = score for assignments using vary_rubric_by_rounds feature
      @review_scores = @assignment.compute_reviews_hash
      @avg_and_ranges= @assignment.compute_avg_and_ranges_hash
    when "FeedbackResponseMap"
      #Example query
      #SELECT distinct reviewer_id FROM response_maps where type = 'FeedbackResponseMap' and 
      #reviewed_object_id in (select id from responses where 
      #map_id in (select id from response_maps where reviewed_object_id = 722 and type = 'ReviewResponseMap'))
      @review_response_map_ids = ResponseMap.select("id").where(["reviewed_object_id = ? and type = ?", @id, 'ReviewResponseMap'])
      @response_ids = Response.select("id").where(["map_id IN (?)", @review_response_map_ids])
      @reviewers = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id IN (?) and type = ?", @response_ids, @type])
    when "TeammateReviewResponseMap"
      #Example query
      #SELECT distinct reviewer_id FROM response_maps where type = 'TeammateReviewResponseMap' and reviewed_object_id = 711
      @reviewers = ResponseMap.select("DISTINCT reviewer_id").where(["reviewed_object_id = ? and type = ?", @id, 'TeammateReviewResponseMap'])
    end
  end

The same code after moving the methods to their respective models looks as follows:

case @type
      when "ReviewResponseMap"
        @review_user= params[:user]
        #If review response is required call review_response_report method in review_response_map model
        @reviewers = ReviewResponseMap.review_response_report(@id, @assignment,@type, @review_user)
        @review_scores = @assignment.compute_reviews_hash
        @avg_and_ranges= @assignment.compute_avg_and_ranges_hash
      when "FeedbackResponseMap"
        #If review report for feedback is required call feedback_response_report method in feedback_review_response_map model
        @reviewers = FeedbackResponseMap.feedback_response_report(@id, @type)
      when "TeammateReviewResponseMap"
        #If review report for teammate is required call teammate_response_report method in teammate_review_response_map model
        @reviewers = TeammateReviewResponseMap.teammate_response_report(@id)
    end
  end

Testing UI

Peer review information:

  • Instructor Login:
       Username: Instructor6
       Password: password
  • Student login:
       Username: Student6384
       Password: password

Steps for testing UI:

  1. Login as an instructor (Using Instructor6 will help you import the participants from other assignments).
  2. Navigate to "Manage->Assignments".
  3. Click on "New Public Assignment" for creating a new assignment.
  4. Create a new assignment by providing assignment name, selecting a course, submission directory (Give any name) and description URL.
  5. Select "has teams?" and provide the team size. Click on create to create a new assignment.
    1. After that, click on review strategy and limit the number of reviews per submission.
    2. Click on "Due dates" and update date for submission and review. Adjust the review date and time in order to test the reviews.
    3. Click on "save". A new assignment is created and it can be viewed in "Manage->Assignments" section.
  6. In order to add participants, there are two methods to add students to the assignment.
    1. Click on "add participants" against the assignment. Enter the user login to add the student. Add atleast 6 participants so that the review mapping can be seen.
    2. Click on any previous assignment "add participants" and export the students list (I used wiki assignment).
    3. Click on your assignment "add participants" and import the students using the export file.
  7. Go back to "Assignments" section and click against "create teams".
    1. After clicking on "create teams", Click on "create teams" in the directed page.
      1. Teams can be formed either manually or automatically.
  8. Login as a student to submit the links for assignment (Valid hyperlink must be provided in the submission link). Add submission for atleast 5 students in order to check the automatic review mapping. (Password for student is password)
  9. After submitting the links for some students, Log in as an instructor to change the assignment phase to review phase.
    1. To change the review phase period, Go to assignments section and click on edit. Click on due dates and change the review due date.
  10. Now, login as a student and go to others work. Click on "Request a submission to review" and check whether a review is assigned automatically. If no assignments are submitted, then this cannot be tested.
  11. For teammate review report, author feedback report and review report, click against "review report" and all the review reports can be seen by selecting it in the drop down menu.

References

  1. Expertiza on GitHub
  2. Forked repository for the project
  3. Expertiza Main Page