E1867 allow reviewer to say review can be shown to class as an example

Problem Statement

The objective of this project is to:

1. Add a feature for students to make their reviews 'public', that is, allow other students to view them.
2. Add a feature for TA to select a subset of 'public' reviews, and make those reviews visible as sample reviews of the particular assignment.
3. Add a feature for Instructor to select a subset of 'public' reviews and make those reviews visible as sample reviews of any of his/her assignments in the course.
4. Create a view where the student can see a list of sample reviews of the assignment and have a detailed view of each.
5. Allow the student to toggle the visibility of a review he/she has submitted.

Approach & Design

1. Consent to make the review public - add a checkbox and an oncheck event handlers (JS) that sets a new field 'visibility' to public of Response object.
2. Change the schema of Responses table (add the new column) and write db migrations.
3. Create new table, model, view, controller for "similar_assignments" and validate CRUD operations that access the table.
4. Add HTML (checkbox) to uncheck the consent such that the reviews becomes private again.
5. On the "popup/team_users_popup" page (where instructor/TA can view all reviews), we give a checkbox against every review with public visibility to allow instructors/TAs to select one or more reviews as sample reviews to be available for students.
6. Once the instructor selects and submits some reviews as sample reviews, we give a popup containing a list of assignments with a checkbox against each of them and a submit button at the end to allow instructors to make sample reviews available for multiple assignments in one go. We perform validation checks also.
7. On submit of the popup, we update the similar_assignments table.
8. At the top of student_reviews/list page, we give an option for the student to preview all the available sample reviews.
9. Create the MVC setup for this new page to list all the sample reviews. Students will be able to click on one particular review and preview it.

DB Design

Currently, the Expertiza database design does not maintain any link between different assignments. An assignment of the current semester is completely independent of any other assignment of any other course having similar or exact objectives. For example, in a course C, the assignment named A1 in a semester Sx has no association to the same assignment given in an earlier semester Sy.

Since our task involves using reviews from the past as samples for the present, we should create this association as a new table. Further, reviews may not be the only intention to associate assignments with each other, as future requirements might require associating them on other criteria too. Thus, we find it appropriate to name the table as "similar_assignments".

The table structure is defined here.

This table can be visualized as a directed graph where vertices represent assignments. An edge (u,v) with a label L means u is similar to v for the intent of association, L.

Consider a row (id = x, assignment_id = Ak, is_similar_for = An, association_intent = "review"). This means, given the intent of association as "review", assignment Ak was chosen as a similar assignment for assignment An. That is, while marking some review for An as a sample, the instructor opted to have reviews of Ak as samples for An as well.

Implementation Details

UML - Use Case Diagram

Power Users: Instructors, TAs, Admins and Super Admins

More details about each point mentioned in the approach section will be updated after the implementation.

Implementation

We have added a new column in the ‘responses’ table (in the database), namely, ‘visibility’. Visibility can have 4 values as listed below:

Table showing list of visibility values and the properties

Visibility	Properties
0	Response is private. This is the default value.
1	The response which was in the review has been approved as an example (that is, it has been marked as a sample)
2	The response which was in the review has been approved as an example (that is, it has been marked as a sample)
3	The response which was either in the review or approved earlier, has now been rejected as an example (it has been unmarked as a sample)

We have maintained these constants in a new file at helpers/response_constants.rb.

module ResponseConstants
	def _private
		0
	end

	def in_review
		1
	end

	def approved_as_sample
		2
	end

	def rejected_as_sample
		3
	end
end

In the existing implementation of reviews, when a student starts a review (but hasn’t filled any text or ratings yet), a response gets created in the database. This response has a default visibility of 0.

We have provided a checkbox (an HTML input node with attribute ‘type’ as ‘checkbox’) for the student to consent for his / her review to be used as an example, with values 0 and 1 for unchecked and checked respectively. This value is used to update the response object as part of the form submission.

However, a student might want to remove their consent or give their consent, for a review that is already submitted. So, in the page where student views his / her own response, we have added the same HTML checkbox. On checking or unchecking the checkbox, the visibility field in the corresponding response object in the database gets updated.Consider this as case 1.

Now when a power user looks at a review (at url , which is an HTML rendering of a ResponseMap containing Responses), he/she must be able to approve and reject responses which students have consented (offered) to be made visible to all. This is done by providing a button “Mark as Sample” or “Remove as Sample”. Responses without consent will not have any button.

These HTML changes appear in views/popup/team_users_popup.html.haml

if instance_variable_get('@sample_button_'+round.to_s)
        %div{:class=>'col-md-12 mark-delete-sample '+'round'+round.to_s}
          %h3{:class=>'col-md-10'} "Reviewed by #{User.find(@reviewer_id).fullname} (Round #{round})"
          %div{:class=>'col-md-2'}

            %button{:class=>'col-md-12 mark-unmark-sample-button '+instance_variable_get('@mark_as_sample_'+round.to_s),
                   :data=>{'visibility':2, 'response_id':instance_variable_get('@response_id_round_' + round.to_s),'round':round} } Mark as Sample
            %button{:class=>'col-md-12 mark-unmark-sample-button '+instance_variable_get('@remove_as_sample_'+round.to_s),
                   :data=>{'visibility':3, 'response_id':instance_variable_get('@response_id_round_' + round.to_s),'round':round} } Remove as Sample

            %span{:class=>"hide col-md-12 mark-unmark-sample-result", :id=>"mark_unmark_fail_"+round.to_s}
            %span{:class=>"hide col-md-12 mark-unmark-sample-result success", :id=>"mark_unmark_success_"+round.to_s}

Clicking on “Mark as Sample” or “Unmark as Sample” changes the visibility of the response to 2 or 3 respectively. Like case 1 mentioned above, this is another situation where the visibility needs to be updated when there is a page event.

Both these cases are event-based and require to make an HTTP request from the browser and handle the response while staying on the same page. Therefore, according to standard practice, an AJAX request must be initiated on click of the button or checkbox. The request needs two parameters - response id, and the updated value of visibility. User-identifying data is sent by default in HTTP header cookies (as in any usual HTTP request).

To handle these requests, we have created a sample_reviews controller with a method update_visibility. Within the method, we first check whether the request parameters are valid, and then proceed to check whether the current user (identified by session id) should be allowed to edit the response object to the visibility value passed in the request.

We have validated requests in this manner since HTTP is stateless and requests can easily be cloned (a browser need not be the only client making requests). For example, one could simply copy the request as a cURL, change the session id in the HTTP header to a student’s session, and pass alternative values of visibility (say 2, meaning, approved as a sample) in the request. Such requests should be identified as unauthorized requests, while requests which want to update visibility to a value that’s not allowed need to be rejected as bad requests.

def update_visibility
	begin
	visibility = params[:visibility] #response object consists of visibility in string format
	if(visibility.nil?)
	raise StandardError.new("Missing parameter 'visibility'")
        end
	visibility = visibility.to_i

	if not (_private..rejected_as_sample).include? visibility
		raise StandardError.new("Invalid value for parameter 'visibility'")
	end
	@@response_id = params[:id]
	response_map_id = Response.find(@@response_id).map_id
	response_map = ResponseMap.find(response_map_id)
	assignment_id = response_map.reviewed_object_id
	is_admin = [Role.administrator.id,Role.superadministrator.id].include? current_user.role.id
	if is_admin
		Response.update(@@response_id.to_i, :visibility => visibility)
		update_similar_assignment(assignment_id, visibility)
		render json:{"success" => true}
		return
	end
	course_id = Assignment.find(assignment_id).course_id
	instructor_id = Course.find(course_id).instructor_id
	ta_ids = []
	if current_user.role.id == Role.ta.id
	        ta_ids = TaMapping.where(course_id).ids # do this query only if current user is ta
	elsif current_user.role.id == Role.student.id
		# find if this student id is the same as the response reviewer id
		# and that visiblity is 0 or 1 and nothing else.
		# if anything fails, return failure
		if visibility > in_review
		           render json:{"success" => false, "error"=>"Not allowed"}
			        return
			    end
		reviewer_user_id = AssignmentParticipant.find(response_map.reviewer_id).user_id
		if reviewer_user_id != current_user.id
			render json:{"success" => false,"error" => "Unathorized"}
			 	return
		         end

	elsif not ([instructor_id] + ta_ids).include? current_user.id
		render json:{"success" => false,"error" => "Unathorized"}
		return
	end
	Response.update(@@response_id.to_i, :visibility => visibility)
	update_similar_assignment(assignment_id, visibility)
		rescue StandardError => e
		render json:{"success" => false,"error" => e.message}
		else
			render json:{"success" => true}
	end
end

We have provided a success or error message text below the button to indicate to the user that the request is completed.

We have used the same request mark_unmark/:id with POST data as {“visibility”:x}, in the student view as well.

This project also required that assignments be linked to each other, as explained in the sections above. To this end, we have created a controller, model, and a migration (to create the database table) for similar assignments.

By default, every assignment is similar to itself (assume it’s id is X). So, after updating a response’s visibility, we add a new entry in the similar_assignments table: (assignment_id = X, similar_for = X, association_intent = “Review”). This addition is done only after confirming for non-existence, as one can infer from the following code snippet:

private
	def update_similar_assignment(assignment_id, visibility)
		if visibility == approved_as_sample
			ids = SimilarAssignment.where(:is_similar_for => assignment_id, :association_intent => intent_review, 
				:assignment_id => assignment_id).ids
			if ids.empty?
				SimilarAssignment.create({:is_similar_for => assignment_id, :association_intent => intent_review, 
				:assignment_id => assignment_id})
			end
		end
		if visibility == rejected_as_sample or visibility == _private
			response_map_ids = ResponseMap.where(:reviewed_object_id => assignment_id).ids
			response_ids = Response.where(:map_id => response_map_ids, :visibility => approved_as_sample)
			if response_ids.empty?
				SimilarAssignment.where(:assignment_id => assignment_id).destroy_all
			end
		end
	end

The power user visits a review response page expecting to mark some reviews as samples and then link or unlink the assignment associated with the review to his / her other assignments. To provide the linking functionality, we have provided a clickable text “ Use as Samples for More Assignments” at the top right corner of the page as shown below.

The appearance or non-appearance of this text is conditional. It only appears when there is at least one sample review for the current assignment. So, to decide this, we have queried for those assignments for which this assignment is a similar one, and checked on the size of the query result. The size is then checked in popup_controller to set an instance variable popup_show to a boolean value. The view uses this to render or not render the clickable “Use as Samples for More Assignments” text.

Code snippet of get_similar_assignment_ids

  def get_similar_assignment_ids(assignment_id)
      SimilarAssignment.where(:assignment_id => assignment_id).pluck(:is_similar_for)
  end

Code snippet of popup_controller line 84

@page_size = popup_page_size

On click of this text, we open a popup that lists out (with a checkbox alongside each list entry) the candidate assignments that this assignment can be similar for (these are unchecked), and assignments that it is already similar for (these are checked). For example, assume that the power user is viewing the response page of assignment X. Assume we have entries (X,Y,”review”) and (X,Z,”review”) in similar_assignments table. Then, the popup list will have the assignments Y and Z with a checkbox against each of them ticked, and all the other assignments that the current user has access to, with a checkbox against each of them unticked.

The popup and the data that go with it are fetched on demand. So, the assignment list need not be a part of the first render of the page itself, as the inherent querying would slow down page load time. Therefore, on click, we fetch the list by an AJAX request (we have given appropriate error messages below the clickable text for the rare eventuality of any server-side error or empty response).

This request GET “/similar_assignments/:id” is again authenticated in similar_assignments_controller, get method. Same validation ideas as explained earlier are used. Only the assignment’s course instructor and TAs, admin, and super-admin should be able to fetch this data, while others need not have access to it.

Code snippet of get method

 # GET /similar_assignments/:id
def get
    @assignment_id = params[:id]
    ids = Response.joins("INNER JOIN response_maps ON response_maps.id = responses.map_id 
    WHERE visibility=2 AND reviewed_object_id ="+@assignment_id.to_s).ids
    if ids.empty?
    render json: {"success"=>false, "error"=>"Please mark atleast one review as sample"}
      return
    end
    begin
      if current_user.role.id == Role.student.id
        throw Exception.new
      end
    @similar_assignments = SimilarAssignment.where(:assignment_id => @assignment_id).
    where.not(:is_similar_for=>@assignment_id).order("created_at DESC").pluck(:is_similar_for)
    rescue ActiveRecord::RecordNotFound => e
      render json: {"success" => false, "error" => "Resource not found"}
    rescue Exception
      render json: {"success" => false, "error" => "Unauthorized"}
    else
      @res = get_asssignments_set(@similar_assignments)
      render json: {"success" => true, "values" => @res}
    end
  end

The get method fetches similar assignment ids and passes them on to get_asssignments_set of similar_assignments_helper. This method queries for all assignments (except the ones passed in its parameter list) that the current user is allowed to access and returns a list of hash objects which are used by the caller (the on-success function of the AJAX request). Each hash object contains assignment name, id, it’s course name, and whether it’s a similar assignment or not. Following code snippets demonstrates this:

Code snippet of get_asssignments_set method

def get_asssignments_set(selected)
    all_assignments = get_assignments_based_on_role
    assignment_array = []
    courses = get_courses_based_on_role
    all_assignments.each {
        |assignment|
      course_id = assignment.course_id
      if(course_id.nil?)
       next
      end
      if (selected.include? assignment.id)
        hash1 = {:title => assignment.name, :course_name => courses[course_id], :checked => true, :id => assignment.id}
        assignment_array.push(hash1)
      else
        hash2 = {:title => assignment.name, :course_name => courses[course_id], :checked => false, :id => assignment.id}
        assignment_array.push(hash2)
      end
    }
    return assignment_array
  end

Code snippet of get_assignments_based_on_role method

def get_assignments_based_on_role()
    role = current_user.role.id
    page = params[:page]
    assignment_id = params[:id].to_i
    if page.nil?
      page = 0
    end
    _offset = page.to_i * popup_page_size
    case role
      when Role.ta.id
        course_ids = TaMapping.where(:ta_id => current_user.id).pluck(:course_id)
        @all_assignments = Assignment.where(:course_id => course_ids).
          where.not(:id=>assignment_id).limit(popup_page_size).offset(_offset).order("created_at DESC")
      when Role.instructor.id
        course_ids = Course.where(:instructor_id => current_user.id).pluck(:id)
        @all_assignments = Assignment.where(:course_id => course_ids).
           where.not(:id=>assignment_id).limit(popup_page_size).offset(_offset).order("created_at DESC")
      else
        @all_assignments = []
    end
    @all_assignments
  end

The query result can be prohibitively large. This calls for pagination.

Pagination at the server side is about performing the same query as a non-pagination implementation, with an addition: limit and offset. Limit denotes the size of a page, and offset denotes a start index, and is calculated as page size * page number. Page size is a constant defined in similar_assignments_constants.

We had two options for response and rendering: Construct the popup list HTML from the query results and respond with this HTML. Respond with JSON, and build the HTML client side.

Responding with HTML from server-side is expensive, as the byte size of the HTTP response would be significantly larger than JSON. Thus, we chose the second option.

Since HTTP is stateless, the server has no way to maintain a running counter of page number. Therefore, keeping track of page number must be the responsibility of the client (browser JavaScript) who must pass it as a request parameter while the server simply reads it and assumes its lack of existence as page number 0. The following snippet demonstrates this:

Snippet of response.js line 103-130

fetchAssignments:function(){
    var self = this;
    var ajaxUrl = "/similar_assignments/"+assignmentId;
    if(this.currentPageNumber > 0){
        ajaxUrl += "?page="+this.currentPageNumber;
    }
    jQuery.ajax({
        "url":ajaxUrl,
        "type":"GET",
        "dataType":"json",
        "responseType":"json",
        "beforeSend":function(){
            self.hideSuccess();
            self.hideError();
                    },
        "success":function(result){
            if(result.success && result.values.length){
                self.onFetchSuccess(result.values);
            }else if(result.success && self.currentPageNumber == 0){
                self.onFetchFail("Cannot link this to any assignment!");
            }else if(result.success){
                self.onFetchSuccess(result.values);
            }
            else{
            self.onFetchFail(result.error);
            }
        },
        "failure": self.onFetchFail,
        "error":self.onFetchFail

We have avoided maintaining the constant value of page size once on server side (in similar_assignments_constants) and once on the client side (JS variable). The initial render of the page creates a hidden HTML element that contains the page size, and once the document is ready, response.js selects that element, reads its value into a variable, and deletes that HTML element. This prevents having a page size in the HTML page-source, and more importantly, complies with the DRY principle. Page size should be changed in one function only - and that is in similar_assignments_constants as explained earlier.

From the response JSON containing an array of objects (that are going to be rendered as list items), we have constructed the HTML list. Each list item maintains its assignment id in its HTML data attribute. The following snippet shows this.

Response.js lines 67-78

addToList:function(assignment){
        var assignmentId = assignment.id;
        var title = assignment.title;
        var course = assignment.course_name;
        var displayText = "<strong>"+course+": </strong>" +title;
        var checked = assignment.checked;
        var newRow = this.template.clone();
        newRow.removeClass("hide").find("input").attr("data-id",assignmentId).attr("checked",checked);
        newRow.find("span").html(displayText).on("click",function(){
            newRow.find("input").click();
        });
        this.list.append(newRow.removeAttr("id"));
    },

The “More” button (shown in the snapshot above) is used to fetch the next set of results.

After a finite number of requests, the client receives a lesser number of results than expected. This is when pagination must stop (since there are no more results). This is achieved by the object AssignmentsPopup (in our response.js file) - it maintains the page size and current page number. The more button is now replaced by a text “No more results” and it’s on-click functionality is turned off.

Further, we have implemented this as a plug-and-play service. By reusing the same popup HTML and the AssignmentsPopup variable, the entire popup can be made a part of any Expertiza page.

The page makes no requests on closing the popup, or on clicking Submit without checking any unchecked list item or unchecking any checked list item. It submits only if there is an overall difference in what items are checked and what are not.

Click of “Submit” makes another AJAX POST request to “/similar_assignments/create/:id” with two params in the post JSON data: “checked” - an array of assignment ids that were checked in the list but unchecked while opening the popup “unchecked” - an array of assignment ids that were unchecked in the list but checked while opening the popup

Either of these params may be empty, depending on the user’s actions, but not both (if both were empty, we simply close the popup without submitting). However, this optimization is only client side. Server side logic must still check for duplicates - that is, an assignment id under “checked” is already a part of similar_assignments.

Let current assignment id be X. For each of the checked assignment ids Ci in {C1,C2,...,Cn}, we have checked if there are entries (X,Ci) in similar_assignments table, and if not, then created that entry. Similarly, for each of the unchecked assignment ids Ui in {U1,U2,...,Um}, we have deleted the entry (X,Ui) from similar_assignments.

On a successful update, the client closes the popup and prints a success message.

So far, power users have marked reviews as samples and linked assignments. Now, students need to view these samples. A student would view these sample reviews before writing their review on somebody else’s work. Therefore, we have added a link “View Sample Reviews” to view the list of sample reviews.

Similarly, power users might also want to view samples. We have provided the same link on the following page too.

This page is rendered by the index method in sample_reviews_controller. It is a list of links to sample reviews. The layout of this page is common for users, as long as they are logged in. Non-logged-in or anonymous visitors are redirected to the Expertiza home page. ‘index’ calls a method redirect_anonymous_user in sample_reviews_helper to achieve this.

Snippet of sample_reviews_helper

def redirect_anonymous_user
    current_user = session[:user]
    if current_user.nil?
        redirect_to "/"
    end
end

The list of available samples has no upper bound on its size. Yet again, pagination solves this problem. But unlike the case of the popup discussed earlier, the response type differs. A request without a ‘page’ parameter in its URL is a request for the web-page and should, therefore, render HTML, while the same request URL with the ‘page’ parameter is a request for more content, and should, therefore, render JSON. The following snippet from the index method demonstrates this.

Sample_reviews_controller - index method - add the if-else just before return statement

def index
    redirect_anonymous_user
    @assignment_id = params[:id].to_i
    page_number = params[:page].to_i
    if page_number.nil?
        page_number = 0
    end
    @page_size = 8
    similar_assignment_ids = get_similar_assignment_ids(@assignment_id)
    @response_ids = []
    similar_assignment_ids.each do |id|
        _offset = page_number * @page_size
        ids = Response.joins("INNER JOIN response_maps ON response_maps.id = responses.map_id 
        WHERE visibility=2 AND reviewed_object_id = "+id.to_s+
        " ORDER BY responses.created_at LIMIT "+@page_size.to_s+" OFFSET "+_offset.to_s ).ids
        @response_ids += ids
    end
    @links = generate_links(@response_ids)
    if page_number == 0
        @course_assignment_name = get_course_assignment_name(@assignment_id)
    else
        render json: {"success" => true, "sampleReviews" => @links}
    end
end

During this first render itself, we have also identified whether more results can exist. The “More” button only appears if the number of sample review links being rendered is equal to the page size, as shown in a snippet of views/sample_reviews/index.html.erb below.

Code snip of Lines 12-15

<% if @links.size == @page_size %>
    <h5 style="cursor:pointer;text-align: center;" id="more_button">More +</h5>
<% end %>

Clicking the “More” button now makes the same request as the page’s URL, but with the parameter ‘page’. In that case, the response is JSON, which is used by the caller (JavaScript Ajax) to add more rows to the page. If the number of rows is lesser than expected, the More text is changed, its click listener is turned off, and no more requests are made, demonstrated by the snippet of sample_review.js below.

Code snip of Line 25-35

for(var i in result.sampleReviews)
{
    currentNumberOfRows++;
    var review = result.sampleReviews[i];
    var newRow = listTemplate.clone();
    newRow.removeClass("hide").removeAttr("id").find("a").
    html("Sample Review "+currentNumberOfRows).attr("href",review);
    ulList.append(newRow);
}

if(responseSize < pageSize){
self.html("No more results!").off().css({"cursor":"default"});
}
else
{
pageNumber++;
}

Each of the list items on the page is a link, which takes the user to /sample_reviews/:id. On this page, we fetch the question-answer pairs corresponding to the object having the id given in the URL and print it out in the form of a table.

Test Plan

As a Power User (TA/Instructor/Admin/Super Admin)

1. Log in
2. Click on Manage->Assignments
3. Displays a list of Assignments
4. Click View Report/Review for a particular assignment.
5. Displays a list of reviews submitted by students.
6. Click on any review in "team reviewed" column for a particular student.
7. Displays the summary of reviews submitted by that student, with a "Make as sample" button on the right of every review.
8. Click on "Make as sample" for the intended reviews, which opens a popup that displays a list of all assignments that are a part of the instructor's courses.
9. From this list select all assignments for which the review has to be shown as a sample.
10. Click on 'Submit' after selection (this closes the popup).
11. Navigate to view reviews of that particular assignment and click on "Sample Reviews".
12. A new page is opened that lists out all the sample reviews of the assignment.

As a Student

1. Log in.
2. Click on Assignments
3. List of assignments is displayed.
4. Click on any assignment for which the review has to be submitted.
5. Assignment task page is displayed.
6. Click on "Other's work" to open the reviews summary page (at /student_review).
7. Below the heading "Reviews for ...", click on the "Show sample reviews" link.
8. This opens a page where the student can view all sample reviews for that assignment.
9. Use the browser's back button to go back to the Assignment review page.
10. Chose to review any of the teams' assignments that are displayed.
11. Select a team for review and fill in the review.
12. Before submitting the review, select the checkbox that says "I agree to share this review anonymously as an example to the entire class".
13. After clicking on the submit button, the review submitted has been made public.

RSpec Tests

Below are the Rpsec tests that have been added to test the edge cases:

1. If an anonymous user tries to view the sample reviews page he/she will be redirected to home page.
2. If an anonymous user tries to view the details of a particular sample review, he/she will be redirected to home page.
3. If a user tries to view the sample review of a particular assignment for which there are no sample reviews, he/she will be redirected to home page.
4. If a student tries to fetch similar assignments, then he/she will be redirected to the assignments page
5. If a student tries to update any similar assignment, then he/she will be redirected to the assignments page
6. Performed some basic HTML validations -should always give the content-type as text/html"

Additional Links and References

1. Link to the Git Pull Request
2. Expertiza on GitHub
3. GitHub Project Repository Fork
4. The Live Expertiza Website

Team

Amogh Agnihotri Subbanna
Chinmai Kaidabettu Srinivas
Siddu Madhure Jayanna
Suhas Naramballi Gururaja