Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

2017-12-06T04:32:33Z

Asundar2: /* Rspec Test */

== Introduction ==

The expertiza project takes advantage of peer-review among students to allow them to learn from each other. To track the review time that each student spend on each submitted resources is meaningful to instructors to study and improve the teach experience. However, the time tracking of specific content on these resources is not perfect in the expertiza system. The previous team working on E1705 project has solved part of such problems by tracking the active time of windows opened from the submitted links. But for the submitted files which could be downloaded for local review, there is only a rough approximation solution yet. In this project, our team is going to solve this problem as well as improving previous time tracking methods using mature third-party APIs to record link review time more accurately. To accomplish this goal, here are the general solutions we designed and implemented in this project:

*Designed a database schema for logging the time a reviewer spend on a submission;
*Designed and implemented dynamic partial components using Javascript and Ruby to open students’ submissions in new windows to view online;
*Used DOM to control the windows holding review submission files or links, and record the open and close time;
*Modify Review report (views/review_mapping/_review_report.html.erb) to show the total and detailed time spent on each review submissions with visualization.

== Problem Description and Solution ==
Our team is going to solve the submission review time tracking by attempting to open the downloaded files inside popup windows of the web browser in order to record the time on the client side.

=== Design pattern ===

=== Type of submitted files ===
For most of our tasks, there are two types of submitted files.

1. One kind is online links, such as github repo or pull request link, youtube video link, expertiza wiki page link, etc

2. The other one includes files such as pdf, txt, doc, images, etc and other download contents.

=== Database shcema design ===
For this project, we need to design a database table to store all the time that we need to record for each review submission. We would like to name the table as '''review_time'''. For the table, we would use '''map_id''' in table response, which uses '''response_map_id''' as the primary information. Then we need to record '''reviewer_id''' and '''reviewee_id''', '''file_link''' for submitted files, as well as time '''start_at''' and '''end_at''', then store them in the table.

[[File:E1791db.png | center]]

=== Time tracking mechanism ===

For each review, when the reviewer click on one link of the submitted files, we create an entry to the table review_time, we will write a function to record current time to the database.

For how to determine time when the reviewer complete with the opened link, there are different cases. If the reviewer click on save or submit button, close the page, or logout, we need to write to all open entry (without end_at) respect to this review and save current time to end_at. But for how to decide when the review close the opened link is a little complicated. For github and youtube links and txt, pdf, doc, image files, we will open submitted files in views/response/_submitted_files.html.erb so that when the user click on the link, it opens a popup window that shows the submission, record time when it opens and closes. This is done by controlling DOM using jQuery javascript library, and then send back the time records through Ajax. For the other files that need to be downloaded, we can only make an approximation by record the time when the reviewer click on the link until the time he/she enter the reviews.

[[File:E1791_flowchart.png | center]]

=== View modification ===
After we record start_at and end_at time for each piece of review, we need to display time that each reviewer spent for each link. Because the reviewer may save the review and start again in the future, so for each link there might be several entries, we need to sum them together. We will try to modify code in review_mapping/_review_report.html.erb first to modify the view. This framework follows well the "open to extension and close to modification" principle, which makes further changes more feasible by adding additional view components without affecting current functions.
[[File:review_piechart.png | center]]

=== Edge cases ===
1. When user click on save or submit the review without closing opened windows, the system will search all records of response time for the reviewer and reviewee where end time is NULL, and update it with current time.

2. If the reviewer opens the reference links or download files, but leave them open and turn to focus other unrelated stuff, the review page will send a popup alert dialog to confirm if the review is still in review mode. If not, it will close these windows and save the time as approximation of end time. Otherwise, it will continue the review process.
[[File:Alert_popup.png | center]]

== Test Plan ==
=== UI and Manual test ===
We will test the following cases during our development

1. When we work the review, if we click on GitHub link, GitHub pull request link, check whether the system record review time correctly.

2. When we work the review, if we click on doc, pdf, image link, check whether system open submitted files in views/response/_submitted_files.html.erb and recore the review time correctly.

3. Check if the reviewer save the process and start again with click on same clink, check whether the system will record all the entry, and on the view page take all entries for one link into account.

=== Rspec Test ===
Test of new model and function will be written and tested during project development:
* Test the success of a review time record given valid start and end time;

describe ResponseTimesController do
describe '#action_allowed?' do
it "should return true" do
expect(true).to be_truthy
end
end

* Test whether the start time is updated to the end time last saved in the database when the user resumes a review;

describe '#response_start_time' do
context 'when the link is opened and timed' do
it 'should update time record with start time as end time' do
response_time_records=double('ResponseTime')
allow(ResponseTime).to receive(:where).with([:map_id,:round,:link]).and_return(response_time_records)
dummy = double('BasicObject')
allow(ResponseTime).to receive(:end_at).and_return(dummy)
allow(dummy).to receive(:nil?).and_return(true)
allow(ResponseTime).to receive(:update_attributes).with(:end_at,:start_at).and_return(response_time_records)
expect(response.body).to be_blank
end
end
end

* Test whether the end time is updated as the current time when the review is saved to the database.

describe '#response_end_time' do
context 'when response does not have a end time' do
it 'should update time record with end time as current time' do
response_time_records=double('ResponseTime')
allow(ResponseTime).to receive(:where).with([:map_id,:round,:link]).and_return(response_time_records)
dummy = double('BasicObject')
allow(ResponseTime).to receive(:end_at).and_return(dummy)
allow(dummy).to receive(:nil?).and_return(true)
allow(ResponseTime).to receive(:update_attributes).with(:end_at,Time.now.to_date).and_return(response_time_records)
expect(response.body).to be_blank
end
end
end

== Related Files ==
review_time.rb <br>
responses_controller.rb <br>
response_times_controller.rb <br>
helper/review_mapping_helper.rb <br>
response/response.html.erb <br>
review_mapping/_review_report.html.erb <br>
review_mapping/_review_submissions_time_spent.html.erb <br>
view/popup/reviewer_details_popup.html.erb <br>
view/submitted_content/_hyperlink.html.erb <br>
view/submitted_content/_submitted_files.html.erb <br>
db/migrate/20171117190721_create_response_times.rb <br>
config/routes.rb

== Future Work ==

In this project, we accomplished the requirement defined in the project instructions. However, due to the complexity of the Expertiza system and limited time period, there are still several points to improve in the future:

1. Improve the accuracy of review time record. To record the exact time that the user spends on each link or file, it is more accurate to track the active time of each new window or tab of these links or online files. We attempted this design using 'window.focus()', but it does not work once the user moved away from the review page and turned to other applications. So either tweaking the web setting or using browser APIs may solve this problem.

2. Notify the user when no response on the review page. Occasionally the user may open the review page to do the review, but is interrupted to focus on other applications. In this case, we stopped the time recording but try to alert the user that if he wants to continue the review. However, this notice cannot show in time if the user is not focusing on that review page. So a global alert is necessary to save the time and provide notice on time.

== Project Links ==
1. [https://github.com/expertiza/expertiza/pull/1124 Project Github repo]

2. [https://youtu.be/IMTBqiKth-k Project Video Demo]

==Team==
[mailto:sli41@ncsu.edu Shijie Li]<br>
[mailto:wsun12@ncsu.edu Wei Sun]<br>
[mailto:asundar2@ncsu.edu Aishwarya Sundararajan]<br>
[mailto:dbhanda@ncsu.edu Darshan Balakrishna Bhandari]<br>

CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

2017-11-30T17:47:23Z

Asundar2: /* Rspec Test */

CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

2017-11-28T22:40:37Z

Asundar2: /* Related Files */

CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

2017-11-28T22:31:26Z

Asundar2: /* Related Files */

CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement

2017-11-28T22:12:44Z

Asundar2: /* Database shcema design */

CSC/ECE 517 Fall 2017/E1774 Metareview fixes and improvements.rb

2017-11-03T00:03:21Z

Asundar2:

==Introduction==
==='''Problem Statement'''===
In Expertiza, the metareview feature in Expertiza enables a third party to evaluate the quality of feedback that a reviewer gives to a reviewee. Metareview is an important feature that can help students become a better reviewer for their peers by providing detailed feedback that is actionable and positively formulated.
Unfortunately, this feature is broken and the following issues were identified. Your task is to fix these issues so that this feature can be used again.

----
==='''Team Members'''===
The team members who worked on this particular problem are:-

Rachit Agarwal

Cherukeshi Machan

Aishwarya Sundararajan

=='''Issues'''==

==='''[https://github.com/expertiza/expertiza/issues/970 Issue #970] '''===
Dynamic assignment of metareviews does not work. When requesting a new metareview to perform, it shows NoMethodError undefined method ‘includes?’ in assignment.rb line 102.

Fix:
This line has been fixed and now checks if response_map.reviewer == metareviewer rather than response_map.reviewer.includes?(metareviewer), which was invalid.

<pre>
response_map.reviewee == metareviewer or response_map.reviewer == metareviewer
</pre>

==='''[https://github.com/expertiza/expertiza/issues/298 Issue #298]'''===
New Feature: Metareviewer should be able to see author feedback. Currently, the metareviewer can only see the review and doesn't have access to the author feedback.

Implementation: This feature is implemented using the "reviewed_object_id" from the "response_maps" table to fetch the authors feedback for the particular reviewed id. Below are the code changes for implementing this feature.

responses.html
<pre>
<% if map.type.to_s == "MetareviewResponseMap" %>
<td> <%= link_to "Author's Feedback", {controller: 'student_review', action: 'show_authors_feedback',:id => map.reviewed_object_id}%></td>
<% end %>
</pre>

student_review_controller.rb
<pre>
def show_authors_feedback
@map = FeedbackResponseMap.find_by(reviewed_object_id: params[:id])
end
</pre>

show_authors_feedback.html
<pre>
<% if @map and !@map.response.empty? then %>
<%= @map.response.last.display_as_html %>
<%else%>
<h3>Feedback from author</h3>
<h4> No feedback done yet.</h4>
<%end%>
<a href="javascript:window.history.back()">Back</a>
</pre>

==='''[https://github.com/expertiza/expertiza/issues/198 Issue #198] '''===

Meta-reviews do not show the review to be reviewed. When I am assigned the meta-review, the original review I'm supposed to critique is not shown, only the link to the wiki article and the review rubric.

Fix:
Updated response.html.erb: Previously, this view was rendering a partial which was actually not present in the views. So we are not rendering that partial anymore.
Now the actual review is displayed by calling an action of response.controller to show the original review which metareviewer is supposed to critique.

Old Code
<pre>
<% if @responses_versions.empty? %>
<I>No previous review was performed.</I><br/><hr/><br/>
<% else %>
<%= render :partial => 'review', :locals => {:versions => @responses_versions} %>
<% end %>
</pre>

New Code
<pre>
<% if @responses_versions.empty? %>
<I>No previous review was performed.</I><br/><hr/><br/>
<% else %>
<%= @responses_versions.each do |response| %>
<%= response.display_as_html() %>
<br/>
<% end %>
<hr/>
<br/>
<% end %>
</pre>

==='''[https://github.com/expertiza/expertiza/issues/162 Issue #162]'''===
Student selected a new meta review they get their own project to metareview

This functionality is working fine as per the requirement. When a user selects a new metareview, a review is assigned to the user to be reviewed, which have the different author other than the User. This can be tested using the test scenario mentioned in the testing section.

==='''[https://github.com/expertiza/expertiza/issues/145 Issue #145]'''===
Metareviews need to be fixed to work with staggered-deadline assignments

Fix:
The submission is reviewable if the submission is in a stage where reviews can be done. It does not depend on the stage of the topic the reviewer is writing on. The same has been incorporated by the below changes.

Code mentioned in defect:
<pre>
<% if @assignment.review_allowed(@reviewee_topic_id) or @assignment.metareview_allowed(nil) or @assignment.get_current_stage(@participant.topic_id) == "Complete" %>
</pre>

Updated code:

student_task/view.html.erb
<pre>
<% if check_reviewable_topics(@assignment) or @assignment.metareview_allowed(nil) or @assignment.get_current_stage(@topic_id) == "Finished" %>
</pre>

student_task_helper.rb
<pre>
def check_reviewable_topics(assignment)
return true if !assignment.has_topics? and assignment.get_current_stage != "submission"
sign_up_topics = SignUpTopic.where(assignment_id: assignment.id)
sign_up_topics.each {|topic| return true if assignment.can_review(topic.id) }
false
end
</pre>

=='''Test Plan'''==

==='''Automated'''===

===='''Issue #970 '''====

<pre>
describe "GET #new_feedback" do
it "redirects to new if review object is found" do
allow(Response).to receive(:find).and_return(review)
allow(session[:user]).to receive(:id).and_return(1)
allow(review).to receive_message_chain(:map, :assignment, :id).and_return(1)
allow(review).to receive_message_chain(:map, :reviewer, :id).and_return(1)
allow_any_instance_of(AssignmentParticipant).to receive_message_chain(:where, :first).and_return(assignment)
allow_any_instance_of(FeedbackResponseMap).to receive_message_chain(:where, :first).and_return(map)
allow_any_instance_of(FeedbackResponseMap).to receive(:create).and_return(map)

get :new_feedback

expect(response).to redirect_to action: :new, id: map.id, return: "feedback"
end
it "redirects to same page if no review is found" do
allow(Response).to receive(:find).and_return(false)
expect(response).to have_http_status(200)
end
end
</pre>

===='''Issue #298'''====
We wrote test for Issue "Metareviewer should be able to see author feedback". student_review_controller_spec.rb file has been created

Code:
<pre>
RSpec.describe StudentReviewController do
context 'logged in as metareviewer' do
let(:review) { Response.create(map_id: 1, additional_comment: 'hello', round: 1) }
let(:map) { FeedbackResponseMap.create(reviewed_object_id: 1, reviewer_id: 1, reviewee_id: 1) }
let(:assignment) { AssignmentParticipant.new }
let(:responsemap) { ResponseMap.new }

describe "GET #authorFeedback" do
it "returns authors feedback" do
get :show_authors_Feedback
expect(response).to redirect_to(request.env['HTTP_REFERER'] ? :back : :root)

end
end

end

end
</pre>

<pre>
StudentReviewController logged in as metareviewer GET #authorFeedback returns authors feedback
0.21359 seconds ./spec/controllers/student_review_controller_spec.rb:12

1 example, 0 failures, 1 passed

Finished in 0.563879969 seconds

</pre>

===='''Issue #198'''====

==='''Manual '''===

===='''Issue #970 '''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

===='''Issue #298'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) Option to view the author's feedback will be visible next to Begin button.

===='''Issue #198'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) click begin button and show review option should be visible on the top to view the origin review to be reviewed.

===='''Issue #162'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) Select begin button.

8) The submitted work is different from the work submitted by the metareviewer.

===='''Issue #145'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select "Staggered deadline assignment?" checkbox in general tab and update the deadlines for metareview in "due dates" tab.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) "Request a new metareview" option should be visible to the user if the deadline is the future date for that particular assignment.

==References==

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]

CSC/ECE 517 Fall 2017/E1774 Metareview fixes and improvements.rb

2017-11-03T00:02:12Z

Asundar2: /* Issue #298 */

==Introduction==
==='''Problem Statement'''===
In Expertiza, the metareview feature in Expertiza enables a third party to evaluate the quality of feedback that a reviewer gives to a reviewee. Metareview is an important feature that can help students become a better reviewer for their peers by providing detailed feedback that is actionable and positively formulated.
Unfortunately, this feature is broken and the following issues were identified. Your task is to fix these issues so that this feature can be used again.

----
==='''Team Members'''===
The team members who worked on this particular problem are:-

Rachit Agarwal

Cherukeshi Machan

Aishwarya Sundararajan

=='''Issues'''==

==='''[https://github.com/expertiza/expertiza/issues/970 Issue #970] '''===
Dynamic assignment of metareviews does not work. When requesting a new metareview to perform, it shows NoMethodError undefined method ‘includes?’ in assignment.rb line 102.

Fix:
This line has been fixed and now checks if response_map.reviewer == metareviewer rather than response_map.reviewer.includes?(metareviewer), which was invalid.

<pre>
response_map.reviewee == metareviewer or response_map.reviewer == metareviewer
</pre>

==='''[https://github.com/expertiza/expertiza/issues/298 Issue #298]'''===
New Feature: Metareviewer should be able to see author feedback. Currently, the metareviewer can only see the review and doesn't have access to the author feedback.

Implementation: This feature is implemented using the "reviewed_object_id" from the "response_maps" table to fetch the authors feedback for the particular reviewed id. Below are the code changes for implementing this feature.

responses.html
<pre>
<% if map.type.to_s == "MetareviewResponseMap" %>
<td> <%= link_to "Author's Feedback", {controller: 'student_review', action: 'show_authors_feedback',:id => map.reviewed_object_id}%></td>
<% end %>
</pre>

student_review_controller.rb
<pre>
def show_authors_feedback
@map = FeedbackResponseMap.find_by(reviewed_object_id: params[:id])
end
</pre>

show_authors_feedback.html
<pre>
<% if @map and !@map.response.empty? then %>
<%= @map.response.last.display_as_html %>
<%else%>
<h3>Feedback from author</h3>
<h4> No feedback done yet.</h4>
<%end%>
<a href="javascript:window.history.back()">Back</a>
</pre>

==='''[https://github.com/expertiza/expertiza/issues/198 Issue #198] '''===

Meta-reviews do not show the review to be reviewed. When I am assigned the meta-review, the original review I'm supposed to critique is not shown, only the link to the wiki article and the review rubric.

Fix:
Updated response.html.erb: Previously, this view was rendering a partial which was actually not present in the views. So we are not rendering that partial anymore.
Now the actual review is displayed by calling an action of response.controller to show the original review which metareviewer is supposed to critique.

Old Code
<pre>
<% if @responses_versions.empty? %>
<I>No previous review was performed.</I><br/><hr/><br/>
<% else %>
<%= render :partial => 'review', :locals => {:versions => @responses_versions} %>
<% end %>
</pre>

New Code
<pre>
<% if @responses_versions.empty? %>
<I>No previous review was performed.</I><br/><hr/><br/>
<% else %>
<%= @responses_versions.each do |response| %>
<%= response.display_as_html() %>
<br/>
<% end %>
<hr/>
<br/>
<% end %>
</pre>

==='''[https://github.com/expertiza/expertiza/issues/162 Issue #162]'''===
Student selected a new meta review they get their own project to metareview

This functionality is working fine as per the requirement. When a user selects a new metareview, a review is assigned to the user to be reviewed, which have the different author other than the User. This can be tested using the test scenario mentioned in the testing section.

==='''[https://github.com/expertiza/expertiza/issues/145 Issue #145]'''===
Metareviews need to be fixed to work with staggered-deadline assignments

Fix:
The submission is reviewable if the submission is in a stage where reviews can be done. It does not depend on the stage of the topic the reviewer is writing on. The same has been incorporated by the below changes.

Code mentioned in defect:
<pre>
<% if @assignment.review_allowed(@reviewee_topic_id) or @assignment.metareview_allowed(nil) or @assignment.get_current_stage(@participant.topic_id) == "Complete" %>
</pre>

Updated code:

student_task/view.html.erb
<pre>
<% if check_reviewable_topics(@assignment) or @assignment.metareview_allowed(nil) or @assignment.get_current_stage(@topic_id) == "Finished" %>
</pre>

student_task_helper.rb
<pre>
def check_reviewable_topics(assignment)
return true if !assignment.has_topics? and assignment.get_current_stage != "submission"
sign_up_topics = SignUpTopic.where(assignment_id: assignment.id)
sign_up_topics.each {|topic| return true if assignment.can_review(topic.id) }
false
end
</pre>

=='''Test Plan'''==

==='''Automated'''===

===='''Issue #970 '''====

<pre>
describe "GET #new_feedback" do
it "redirects to new if review object is found" do
allow(Response).to receive(:find).and_return(review)
allow(session[:user]).to receive(:id).and_return(1)
allow(review).to receive_message_chain(:map, :assignment, :id).and_return(1)
allow(review).to receive_message_chain(:map, :reviewer, :id).and_return(1)
allow_any_instance_of(AssignmentParticipant).to receive_message_chain(:where, :first).and_return(assignment)
allow_any_instance_of(FeedbackResponseMap).to receive_message_chain(:where, :first).and_return(map)
allow_any_instance_of(FeedbackResponseMap).to receive(:create).and_return(map)

get :new_feedback

expect(response).to redirect_to action: :new, id: map.id, return: "feedback"
end
it "redirects to same page if no review is found" do
allow(Response).to receive(:find).and_return(false)
expect(response).to have_http_status(200)
end
end
</pre>

===='''Issue #298'''====
===='''Issue #298'''====
We wrote test for Issue "Metareviewer should be able to see author feedback". student_review_controller_spec.rb file has been created

Code:
<pre>
RSpec.describe StudentReviewController do
context 'logged in as metareviewer' do
let(:review) { Response.create(map_id: 1, additional_comment: 'hello', round: 1) }
let(:map) { FeedbackResponseMap.create(reviewed_object_id: 1, reviewer_id: 1, reviewee_id: 1) }
let(:assignment) { AssignmentParticipant.new }
let(:responsemap) { ResponseMap.new }

describe "GET #authorFeedback" do
it "returns authors feedback" do
get :show_authors_Feedback
expect(response).to redirect_to(request.env['HTTP_REFERER'] ? :back : :root)

end
end

end

end
</pre>

<pre>
StudentReviewController logged in as metareviewer GET #authorFeedback returns authors feedback
0.21359 seconds ./spec/controllers/student_review_controller_spec.rb:12

1 example, 0 failures, 1 passed

Finished in 0.563879969 seconds

</pre>

===='''Issue #198'''====

==='''Manual '''===

===='''Issue #970 '''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

===='''Issue #298'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) Option to view the author's feedback will be visible next to Begin button.

===='''Issue #198'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) click begin button and show review option should be visible on the top to view the origin review to be reviewed.

===='''Issue #162'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select dates and add metareview rubric for the same.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) Select the option "request a new metareview" and a new review will be dynamically assigned for meta-review.

7) Select begin button.

8) The submitted work is different from the work submitted by the metareviewer.

===='''Issue #145'''====

1) Login as Admin, create an assignment with all the necessary details like maximum no of users per team, add topics for the assignment, enroll users(students) in the assignment.

2) Select checkbox "add metareview deadlines", select "Staggered deadline assignment?" checkbox in general tab and update the deadlines for metareview in "due dates" tab.

3) Go back to manage assignment and add select Assign reviewers.

4) Add meta-reviewer to a particular review submitted by students.

5) Impersonate the meta-reviewer assigned above and go to others work for the same assignment.

6) "Request a new metareview" option should be visible to the user if the deadline is the future date for that particular assignment.

==References==

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]