E1875 Revision Planning Tool: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
 
(18 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<br/>
<br/>
== What's it about? ==
== What's it about? ==
In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work.  Then in the second round, reviewers rate how well authors have followed their suggestions.  We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews.  That is, authors would say what they were planning to do to improve their work.  Then second-round reviewers would assess how well they did it.  In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission.  We are interested in having this implemented and used in a class so that we can study its effect.
In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work.  Then in the second round, reviewers rate how well authors have followed their suggestions.  We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews.  That is, authors would say what they were planning to do to improve their work.  Then second-round reviewers would assess how well they did it.  In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission.  We are interested in having this implemented and used in a class so that we can study its effect.  


<br/>
<br/>
Line 10: Line 10:
*** How effectively did feature X address / solve issues a, b and c?
*** How effectively did feature X address / solve issues a, b and c?
*** Did modification of feature Y resolve error d?
*** Did modification of feature Y resolve error d?
* Every new question must be linked to the second-round questionnaire.
* The new questionnaire must be linked to the second-round questionnaire.
* Every new question must be linked to the author’s submission
* The new questionnaire must be part of the team's submission records.


<br/>
<br/>
== Problem Statement ==
== Problem Statement ==
In the 2nd round of reviews, the Author should be able to add a statement to direct towards Author selected improvements from Round 1 to Round 2.
In the 2nd round of reviews, the Author should be able to add a statement to direct towards Author selected improvements from Round 1 to Round 2.
Line 22: Line 23:
* Questions unique to each project gives the reviewers a perspective on the author’s objectives.
* Questions unique to each project gives the reviewers a perspective on the author’s objectives.
* Allow the Author to get feedback on whether or not they accomplished their self-directed goal.
* Allow the Author to get feedback on whether or not they accomplished their self-directed goal.
This project has been extended and reworked under Independent Study in Spring 2019.
<br/>


<br/>
== Criteria for completion ==
== Criteria for completion ==
# Direct user to Revision Improvement Questionnaire.
# Direct user to Revision Planning Questionnaire.
# Create a form for a Assignment Team to add Questions to a Questionnaire that are specific to that Submission.
# Create a form for the Assignment Team to add Questions to a Questionnaire that are specific to that Submission in the second round of submission.
# Append Revision Improvement Questionnaire to 2nd Round Review Questionnaire.
# Append Revision Planning Questionnaire to 2nd Round Review Questionnaire.


<br/>
<br/>
Line 43: Line 45:


==== Controllers ====
==== Controllers ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-a7ee7dddd7aedbc0b814d98f0cb34c25 grades_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1456 questionnaires_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-b41fa84d1ee0c03c4e301f977fbe453d questionnaires_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f response_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-7786ab741930f6bace28b7e5da61b8d6 response_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f submitted_content_controller.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-33d668958529875af7029d78e37aff60 submitted_content_controller.rb]


==== Views ====
==== Views ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-2213cf9074761b4121a9fb3ddcce6145 questionnaires/_questionnaire.html.erb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f questionnaires/_questionnaire.html.erb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-6b83ba9473bd5cc58d3b7c620235a6c0 submitted_content/edit.html.erb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/9bc76076b1739f2d2ddaff74782ea6f1a046f38a submitted_content/edit.html.erb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-6b83ba9473bd5cc58d3b7c620235a6c0 submitted_content/edit.html.erb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/9bc76076b1739f2d2ddaff74782ea6f1a046f38a submitted_content/edit.html.erb]


==== Models ====
==== Models ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-a983bfc492e6d1982c673bc208afe459 answer.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f questionnaire.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-3dffd0131e1a14847e8c98376bae4984 questionnaire.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f response.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-f2caf2e29f94ec94cb27bdb86ec85d40 response.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-85f51952994f8162980c605efd60db44 revision_review_questionnaire.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-c6f2a3b3452f5a9effa4a2b023ed2322 submission_record.rb]


==== Database ====
==== Database ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-c2ba560985d8d1ea9184e04d71bf2c83 20181123004154_add_submission_record_to_questionnaire.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f schema.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-1acd2e7e27a227829d5d14a91c863bb6 schema.rb]


==== Specs ====
==== Specs ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-ebb9ab8d49abf2b40d4aad426dad68e5 grades_controller_spec.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/2bd9590d18b56d0fcc8850f729a00370cd23190f questionnaires_controller_spec.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-df286dc963f50de24c47bacfb0b3a143 questionnaires_controller_spec.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-3a17ddf9d828caa60f11423f20fb88a9 factories.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-7d95010cdfe370a461e216ba1d724392 assignment_submission_spec.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-a68a7982922073b9602c0fd6097e0bd2 answer_spec.rb]
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-e162432910d3b3b29b3c9711d6e818bb revision_review_questionnaire_spec.rb]


==== config ====
==== config ====
* [https://github.com/expertiza/expertiza/pull/1302/files#diff-21497849d8f00507c9c8dcaf6288b136 routes.rb]
* [https://github.com/expertiza/expertiza/pull/1456/commits/9bc76076b1739f2d2ddaff74782ea6f1a046f38a routes.rb]
 


== Test Plan and Demo ==
== Test Plan and Demo ==


=== Test Plan ===
=== Test Plan ===
# Login as 'instructor6' with password 'password'.
# Login as 'super_administrator2' with password 'password'.
# Create 2 new users named 'abc' and 'xyz'.
# Make an assignment with the name 'Assignment1'.  
# Make an assignment with the name 'RPT'. Add abc and xyz as participants to RPT.
# Make the following selections:
# Set 2 rounds of submissions and reviews in RPT.
## Review Strategy: Allow authors to add to rubric.
# impersonate abc.
## Rubric: Rubric varies by round.
# submit for round 1.
## Add participants: Add existing students - student1, student2
# move to round 1 review stage.
## Due Dates: Add deadlines for rounds.
# impersonate xyz.
## Add topics to the assignment
# add review as xyz.
# Set 2 rounds of submissions and reviews.
# move to round 2 submission stage
# Impersonate student1.
# impersonate abc.
# Signup for topic and form team.
# submit a revision plan through the submission page.
# Make submissions in round 1 submission.
# move to round 2 review.
# Move to round 1 review stage.
# impersonate xyz.
# Impersonate student2.
# verify that revision plan wuestions are added to the review and submit the review.
# Make submissions in round 1 and review it.
# impersonate abc.
# Move to round 2 submission stage
# verify that review has been received on the revision plan questions.
# Impersonate student1.
# Submit a revision plan in the 'Your Work' handle.
# Repeat above two steps for student2.
# Move to round 2 review.
# Impersonate student2.
# Verify that revision plan questions are added to the review by student1'team and submit the review.
# impersonate student1.
# Repeat the above two steps for student1.
# Verify that review has been received on the revision plan questions after assignment is finished.
 
===Demo Video===
 
*[https://www.youtube.com/watch?v=t11JQUEqUKg&feature=youtu.be Demo Video]

Latest revision as of 17:09, 8 May 2019


What's it about?

In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then second-round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.


What needs to be done?

  • Develop UI for authors to create new questions to add to the second round-rubric. This should be a form that includes the following:
    • A description of the revision plan. Eg: We will add feature X to address issues a,b and c. We will modify feature Y and expect it to resolve errors d, c and e.
    • One or more questions for every proposed improvement. Example:
      • How effectively did feature X address / solve issues a, b and c?
      • Did modification of feature Y resolve error d?
  • The new questionnaire must be linked to the second-round questionnaire.
  • The new questionnaire must be part of the team's submission records.


Problem Statement

In the 2nd round of reviews, the Author should be able to add a statement to direct towards Author selected improvements from Round 1 to Round 2.


Motivation

The OSS and Final projects are different for every team. From a reviewers perspective, not all questions make sense for all projects. The motivation behind this project is:

  • Questions unique to each project gives the reviewers a perspective on the author’s objectives.
  • Allow the Author to get feedback on whether or not they accomplished their self-directed goal.

This project has been extended and reworked under Independent Study in Spring 2019.

Criteria for completion

  1. Direct user to Revision Planning Questionnaire.
  2. Create a form for the Assignment Team to add Questions to a Questionnaire that are specific to that Submission in the second round of submission.
  3. Append Revision Planning Questionnaire to 2nd Round Review Questionnaire.



UI mockups

The first image shows a mockup of what the Author will see on the submission page to submit new additional questions for review.

Second is a view of what the reviewer will see. It should blend in with the review questions submitted by the instructor for all similar projects.


Files modified

Controllers

Views

Models

Database

Specs

config

Test Plan and Demo

Test Plan

  1. Login as 'super_administrator2' with password 'password'.
  2. Make an assignment with the name 'Assignment1'.
  3. Make the following selections:
    1. Review Strategy: Allow authors to add to rubric.
    2. Rubric: Rubric varies by round.
    3. Add participants: Add existing students - student1, student2
    4. Due Dates: Add deadlines for rounds.
    5. Add topics to the assignment
  4. Set 2 rounds of submissions and reviews.
  5. Impersonate student1.
  6. Signup for topic and form team.
  7. Make submissions in round 1 submission.
  8. Move to round 1 review stage.
  9. Impersonate student2.
  10. Make submissions in round 1 and review it.
  11. Move to round 2 submission stage
  12. Impersonate student1.
  13. Submit a revision plan in the 'Your Work' handle.
  14. Repeat above two steps for student2.
  15. Move to round 2 review.
  16. Impersonate student2.
  17. Verify that revision plan questions are added to the review by student1'team and submit the review.
  18. impersonate student1.
  19. Repeat the above two steps for student1.
  20. Verify that review has been received on the revision plan questions after assignment is finished.

Demo Video