Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T15:44:43Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/14hi8J5Z6jT7leG7G-7_539Imhb-sBxj0/view Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]
* [https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T15:44:05Z

Dmendez: /* Project */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/14hi8J5Z6jT7leG7G-7_539Imhb-sBxj0/view Notes edited]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]
* [https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T15:43:48Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/14hi8J5Z6jT7leG7G-7_539Imhb-sBxj0/view Notes edited]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:28:26Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]
::[https://drive.google.com/file/d/14hi8J5Z6jT7leG7G-7_539Imhb-sBxj0/view Notes edited]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:28:14Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]
::[[https://drive.google.com/file/d/14hi8J5Z6jT7leG7G-7_539Imhb-sBxj0/view Notes edited]]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:19:58Z

Dmendez: /* Project */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Final Projects List]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:09:57Z

Dmendez: /* Project */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]
* [https://docs.google.com/document/d/1GG3nseSRnExHkuu3vOBlFRcOuN3tgwHcfQp9mCzAZ0k/edit Projects pool]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:05:42Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we just retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we initially started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:04:58Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will get the scores by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the scores we retreived and then show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T05:03:21Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how scores are retreived to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Finally, all we have left is to apply the derived formula using the score we retreived and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.

The ''self-review'' scores are obtained by using the ''true'' parameter in all the methods calls (as shown in the diagram), whereas the ''peer-review'' scores are retreived similarly but omiting this parameter.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T00:54:25Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]
[[File: e1980_Img_ch6.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1980 Img ch6.PNG

2019-12-11T00:53:49Z

Dmendez:

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-11T00:01:37Z

Dmendez: /* Manual Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Log in as instructor again. Go and edit the assignment you created. In the Due dates tab, choose a date and time in the past. e.g. yesterday. This will enable peer reviews.
:10. Sign in again as ''student4000'' Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:11. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-10T23:44:42Z

Dmendez: /* Manual Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu. Select any type of Author Feedback.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T02:16:48Z

Dmendez: /* Future Work */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T02:15:31Z

Dmendez: /* Rspec Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases were developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T02:12:51Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

List of files changed:
* app/controllers/grades_controller.rb
* app/controllers/sign_up_sheet_controller.rb
* app/models/assignment_participant.rb
* app/models/author_feedback_questionnaire.rb
* app/models/response_map.rb
* app/models/review_questionnaire.rb
* app/models/teammate_review_questionnaire.rb
* app/models/vm_question_response.rb
* app/models/vm_question_response_row.rb
* app/views/assignments/edit.html.erb
* app/views/assignments/edit/_general.html.erb
* app/views/grades/_participant.html.erb
* app/views/grades/_participant_charts.html.erb
* app/views/grades/_participant_title.html.erb
* app/views/grades/view_team.html.erb
* app/views/reports/_self_review_report.html.erb
* app/views/student_review/list.html.erb
* config/routes.rb
* db/schema.rb
* spec/controllers/grades_controller_spec.rb
* spec/controllers/sign_up_sheet_controller_spec.rb
* spec/features/assignment_creation_spec.rb
* spec/models/assignment_particpant_spec.rb

Next we show the most relevant improvements performed in this project. Feel free to review all the code changes in Github by following the next link:
[https://github.com/expertiza/expertiza/pull/1611/files view file changes in pull request.]

*'''grades_controller.rb'''

[[File: e1984_Img_ch1.PNG]]

*'''assignment_participant.rb'''

[[File: e1984_Img_ch2.PNG]]

*'''response_map.rb'''

[[File: e1984_Img_ch3.PNG]]

*'''vm_question_response.rb'''

[[File: e1984_Img_ch4.PNG]]

[[File: e1984_Img_ch5.PNG]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1984 Img ch5.PNG

2019-12-07T02:12:21Z

Dmendez:

File:E1984 Img ch4.PNG

2019-12-07T02:12:10Z

Dmendez:

File:E1984 Img ch3.PNG

2019-12-07T02:11:58Z

Dmendez:

File:E1984 Img ch2.PNG

2019-12-07T02:11:46Z

Dmendez:

File:E1984 Img ch1.PNG

2019-12-07T02:07:39Z

Dmendez:

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:45:03Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

[https://github.com/expertiza/expertiza/pull/1611/files View file changes.]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:42:04Z

Dmendez: /* Demo Video */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Watch the video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:39:15Z

Dmendez: /* Manual Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1d.PNG]] [[File:e1984_img2d.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1984 img2d.png

2019-12-07T01:39:07Z

Dmendez:

File:E1984 img1d.PNG

2019-12-07T01:38:34Z

Dmendez:

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:35:28Z

Dmendez: /* Manual Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:e1984_img1c.PNG]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1984 img1c.PNG

2019-12-07T01:35:12Z

Dmendez: self review score display

self review score display

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:29:51Z

Dmendez: /* Current Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

[[File:E1984 img7b.png]]

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:28:43Z

Dmendez: /* Current Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

[[File:E1984 img7b.png]]

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1984 img7b.png

2019-12-07T01:28:31Z

Dmendez: self review process

self review process

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T01:27:42Z

Dmendez: /* Current Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

[[File:E1984 img7a.png]]

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

File:E1984 img7a.png

2019-12-07T01:27:07Z

Dmendez: self review process

self review process

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:59:57Z

Dmendez: /* Manual Testing */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo, then follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:59:14Z

Dmendez: /* Demo Video */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go watch to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:59:00Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score for a given assignment:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:58:28Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:56:57Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is done in ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:56:16Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we have retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:55:58Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for'' method located in ''response_map.rb''.

Now that we retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:54:45Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' method located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for''method located in ''response_map.rb''.

Now that we retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:54:08Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score.

The actual computation for the participant scores is done in ''compute_assignment_score'' located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for''method located in ''response_map.rb''.

Now that we retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:53:47Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers, i.e. the peer-review score

The actual computation for the participant scores is done in ''compute_assignment_score'' located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-review score by simply calling the ''get_assessments_for''method located in ''response_map.rb''.

Now that we retreived the peer-review score and self-review score, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:52:09Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given by peers.

The actual computation for the participant scores is done in ''compute_assignment_score'' located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

The ''get_assessments_for'' method will call the ''reviews'' method located also in ''assignment_participant.rb''.

The ''reviews'' method will finally get the self-assesment score by simply calling the ''get_assessments_for''method located in ''response_map.rb''.

Now that we retreived the peer-review scores and self-review scores, all we have left is to apply the derived formula and show the result in the respective views. All this process is carried on ''grades_controllers.rb'', where we started.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:23:27Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess of how code changes work together to implement the self-review score feature.

{| class="wikitable"
|First, we start in the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.

We access the participant scores by calling the ''scores'' method defined in ''assignment_participant.rb''. Here, the ''scores'' method returns scores that the participant has been given.

The actual computation for the participant scores are done in ''compute_assignment_score'' located also in ''assignment_participant.rb''.

Inside ''compute_assignment_score'' we call another method called ''get_assessments_for'' which is located in ''review_questionnare.rb''.

|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-07T00:08:43Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess we followed to implement the self-review score feature.

{| class="wikitable"
|First, we start looking at the ''grades_controllers.rb'' file which contains all the logic for computing an assignment score.
|[[File:E1926_code_flow.png]]
|}

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-06T23:54:50Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess we followed to implement the self-review score feature.

[[File:E1926_code_flow.png]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-06T23:54:34Z

Dmendez: /* Code Changes */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess we followed to implement the self-review feature.

[[File:E1926_code_flow.png]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]

CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades

2019-12-06T23:36:57Z

Dmendez: /* New Process */

==E1984. Improve self-review Link peer review & self-review to derive grades==

This page provides a description of the Expertiza-based Semester project.

__TOC__

===Problem Statement and Overview===

In Expertiza the Instructor can create assignments and check the 'Allow self-review' box for Students. When enabled, each member evaluates his/her submission with the same criteria as the peer review. Currently, these self-review scores are not used to compute the final scores for the assignment. The task here is to improve the self-review system pertaining to give it a weightage in the final scores for the submissions.

[[Image:Self_Reviews_1.png]]

The objectives we achieved for this project are as follows:

*Provide a formula that takes both the peer review score and self-review score into account for calculating the ''composite score''.
*The ''composite score'' should get higher as the ''self-review score'' gets closer to the ''peer-review score''.
*Make sure that the ''peer-review scores'' are not visible before the self-review submission.
*Display the ''composite score'' on the “View Scores” page.
*Display the ''self-review scores'' in the "View Scores" page and in the "Alternate" (heat-map) view of the peer-reviews, highlighting them as a different type of review.

The need for the self-review section is to know how well the students can judge their work and understand how to evaluate their flaws and rectify them. Also, if they can score themselves close to their peers, then it means that they have good self-assessment skills and they are being honest about what they think of their work. According to self assessment rules: students get more points as their self-review get closer to the scores given by their peer reviewers.

===Code Changes===

The following flowchart illustrates the proccess we followed to mamke improvements to the self-review feature.

[[File:E1926_code_flow.png]]

===Use case===
The following diagrams illustrate how the ''Self Assessment'' feture is meant to be used in Expertiza. Basically, we have two actors: on one side we have the Instructor which will create an assignment, add students to it and enable the self-review option. On the other side, students will be able not only to review their peers' work, but also will be required to review their own work in order to view their final scores, if the self-review option is enabled by the instructor.

[[File:E1926_use_case_2.jpg]] [[File:E1926_use_case.jpg]]

*Actors:

:1. Instructor: This actor is responsible for creating assignments and adding students to the assignment.
:2. Student: This actor is responsible for submitting, self-reviewing and viewing the scores.

*Actions
:1. Instructor: Create Assignment, Enable Self-Review.
:2. Student: Login, Add work and Submit, Submit Self-Review, View Scores.

===Current Process===

Currently, Expertiza allows students to submit an assignment and provides a link to self-review their work. Once the self-reviewing is done, the self-review score gets stored in the database and it is not further used in calculating the overall score of the assignment. Thus, students can score themselves higher than what they should actually get when considering the peer-review scores. This helps students in no way. So, to make a productive use of the self-review score and to help students learn self-evaluating themselves, we will make changes in the review score calculation, explained in the section below.

===New Process===

We derived a new formula that takes into account the ''self-review score'' and calculates a ''composite score'', which will be the student's final score:

[[File:e1984_img6a.png]]

More information regarding the derivation of this formula can be obtain from the following document:

::[https://drive.google.com/file/d/19jp7UdLCTdMhLiqsBR_ArU7gIpxYfkx_/view?usp=sharing Notes for deriving the formula]

===Demo Video===
You are welcomed to watch our 5-min demo video where we briefly explain and test the project functionality.

[[File:e1984_img51a.png]] [[https://www.youtube.com/watch?v=oDPdvZeGpmA&feature=youtu.be Go to video!]]

=== Test Plan ===

==== Manual Testing ====
If you wish to try the self-review feature in Expertiza as shown in the demo. Please follow the next steps:
:1. Go to the link of the deployed application: [http://152.46.16.39:8080/ http://152.46.16.39:8080/]
:2. Log in as instructor. Use the credentials: ''instructor6'', ''password''.
:3. Go to ''Manage->Assigments'' and clik the '+' icon on the top right corner to create an new assignment.
::3.1. In General tab: write the assignment title, select any course and type any submission directory.
::3.2. In Rubrics tab: uncheck the tab ''Review rubric varies by round?'' and select ''Design Doc'' for the Review in the Questionnaire drop-down menu.
::3.3. In Review Strategy tab: go to the bottom and check the ''Allow Self Reviews?'' box.
::3.4. In Due dates tab: for Submission deadline, select a date and time greater than the current date and time when creating this assignment. For Review deadline select a date and time grater than the submission deadline.
::3.5. Click on ''Create'' to save and create the assignment.
:4. Go again to ''Manage->Assigments'', find the assignment you created and click on ''Add participants'' (user icon) to add some students.
::4.1. Add ''student3000'' and ''student4000'' to the assignment.
:5. Sign in as student3000. Log out from the instructor6 account or access to the application from a different browser (incognito window or a different explorer). Use credentials: ''student3000'', ''password''.
:6. Go to test assignment and click on ''work''. Submit any link or file.
:7. Click on ''Review my own work'' button and then the ''begin'' link to start the self review. Grade yourself accordingly and make sure to submit the self review when finished.
:8. Repeat steps 5 to 7 for ''student4000''. Use credentials ''student4000'', ''password''.
:9. Go back to the assignment and click on ''Others work''. Click on ''Request a new submission to review'' and review the ''student3000'''s work. Don't forget to submit the review when done.
:10. Sign in again as ''student3000''. Go to the assignment and click on ''Your Scores''. Here you can see the final average peer review score, which is computed by using the proposed formula in this project. In the scores table, you can see your ''peer review score'' and ''self review score'' (cyan color) side by side.

[[File:E1984_img1.png]] [[File:E1984_img2.png]]

==== Peer-review unlock feature ====
Note that in step 7 of the ''Manual Testing'' section, we are immediatly submitting our self review after our work has been submited. This is the ideal workflow, however imagine the situation where a student forgot to submit his/her self-review questionnarie. Then by requirement of this project, and to make sure that every student submit their self review, the ''Your scores'' link will be disabled for the assignment. So students will only be able to review their scores after submitting their self review.

[[File:E1984_img3a.png‎]]

==== Rspec Testing ====
The following Test cases will be developed to ensure a correct implementation:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The composite scores are saved to the database.
* Self-review scores are displayed with peer-review scores.
* Student can only see their own self-review.

We overloaded the ''get_assessments_for'' method with a second parameter which decides wheather the peer-review scores are retreived (false) or only the student self-review (true). Modifications were made accordingly in the ''grades_controller_spec.rb'' and ''assignment_participant_spec.rb'' files.

[[File: E1984_img41b.png]]

=== Future Work ===
The work done in this project can be built upon by future groups. Several areas can be improved upon and/or new functionality can be added.

* Refactoring of the ResponseMap features: accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches. A small addition to the instructor UI may allow them to select from different self-review formulas.
* Add scores from team evaluation (Your team -> Review) to the formula. True peer assesment involves the use of peer-review scores within the team, not only peer-review scores from other teams as currently implemented.

== Team and Resources ==

===Team_AABD:===
*Amit Mandliya ([mailto:amandli@ncsu.edu amandli@ncsu.edu])
*Ankit Manendra ([mailto:amanend@ncsu.edu amanend@ncsu.edu])
*Bharat Sinha ([mailto:bsinha2@ncsu.edu bsinha2@ncsu.edu])
*Daniel Mendez Lozada ([mailto:dmendez@ncsu.edu dmendez@ncsu.edu])

===References===
====Project====
* [https://github.com/expertiza/expertiza/pull/1611 E1984 Pull Request]
* [https://github.com/ankit8011/expertiza/tree/beta E1984 Github Repository]

====Peer Assessment Theory====
* [https://www.researchgate.net/profile/Paul_Vossen/publication/333022439_SCORING_MODELS_FOR_PEER_ASSESSMENT_IN_TEAM-BASED_LEARNING_PROJECTS/links/5cd6d6b4299bf14d958a4b99/SCORING-MODELS-FOR-PEER-ASSESSMENT-IN-TEAM-BASED-LEARNING-PROJECTS.pdf Scoring models for peer assessment in team-based learning projects]
* [http://tinyurl.com/PiTeamMate Tools for enabling developmental feedback and teamwork grading by peer assessment: PeerAssessPro™]