Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T03:20:50Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”
:;Reputation range of Hamer’s algorithm is :
:red value < 0.5
:yellow value is > 0.5 to 1
:orange value is > 1 to 1.5
:light green value is > 1.5 to 2
:green value is > 2

 
The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. This project determines reputation by subtracting the absolute value of the leniency from 1. Additionally, the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].
:;Reputation range of Lauw’s algorithm is
:red value is < 0.2
:yellow value is < 0.2 to 0.4
:orange value is > 0.4 to 0.6
:light green value is > 0.6 to 0.8
:green value is > 0.8

The instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

#Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)
#Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.
#Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.
#On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T03:17:14Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”
:;Reputation range of Hamer’s algorithm is :
:red value < 0.5
:yellow value is > 0.5 to 1
:orange value is > 1 to 1.5
:light green value is > 1.5 to 2
:green value is > 2

 
The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. This project determines A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].
:;Reputation range of Lauw’s algorithm is
:red value is < 0.2
:yellow value is < 0.2 to 0.4
:orange value is > 0.4 to 0.6
:light green value is > 0.6 to 0.8
:green value is > 0.8

The instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

#Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)
#Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.
#Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.
#On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T03:11:36Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”
:;Reputation range of Hamer’s algorithm is :
:red value < 0.5
:yellow value is > 0.5 to 1
:orange value is > 1 to 1.5
:light green value is > 1.5 to 2
:green value is > 2

 
The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].
:;Reputation range of Lauw’s algorithm is
:red value is < 0.2
:yellow value is < 0.2 to 0.4
:orange value is > 0.4 to 0.6
:light green value is > 0.6 to 0.8
:green value is > 0.8

The instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

#Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)
#Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.
#Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.
#On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T03:05:25Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

#Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)
#Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.
#Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.
#On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

;Reputation range of Hamer’s algorithm is :
:red value < 0.5
:yellow value is > 0.5 to 1
:orange value is > 1 to 1.5
:light green value is > 1.5 to 2
:green value is > 2
;Reputation range of Lauw’s algorithm is
:red value is < 0.2
:yellow value is < 0.2 to 0.4
:orange value is > 0.4 to 0.6
:light green value is > 0.6 to 0.8
:green value is > 0.8
*Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:34:54Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:33:37Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====
Confidence ratings are meant to provide objective value to student assigned peer review scores.

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:32:31Z

Bghagar:

This page documents the project to implement confidence ratings for a student peer review grade based on the reputations of the reviewers
__TOC__
===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:29:40Z

Bghagar:

==Show confidence ratings for grade, based on reputations of reviewers==

===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===The E1625 Project===
====Motivation====

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

====Description of confidence rating and our system====

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

====Design Pattern - Model-view-controller(MVC)====

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

====Documentation====

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Implementation===
====Classes Involved====

====Files Involved====

====Models====

participant

====Views====

grades/view.html.erb

grades/view_my_scores.html.erb

====Controllers====

Reputation_web_service_controller.rb

====Standards====

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

====Tasks Performed====

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:27:21Z

Bghagar:

==Show confidence ratings for grade, based on reputations of reviewers==

===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===Motivation===

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

===Description of confidence rating and our system===

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

===Design Pattern - Model-view-controller(MVC)===

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

===Documentation===

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Classes Involved===

===Files Involved===

===Models===

participant

===Views===

grades/view.html.erb

grades/view_my_scores.html.erb

===Controllers===

Reputation_web_service_controller.rb

===Standards===

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

===Tasks Performed===

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

CSC/ECE 517 Spring 2016

2016-04-04T02:25:06Z

Bghagar: Added E1625 team

==Writing Assignment 1==
*[[CSC/ECE 517 Spring 2016/Active Job]]

==Calibration Assignment Submissions==
*[[Calibration Assignment Submission (OmniAuth)]]
*[[Calibration Assignment Submission (Patch_verb)]]

==Writing Assignment 2==
*[[CSC/ECE 517 Spring 2016/Mozilla Implement HTML5 form validation]]

*[[CSC/ECE 517 Spring 2016/Functional Tests for Questionnaire Controller]]

*[[CSC/ECE 517 Spring 2016 / Expertiza Self-Review Feature]]

*[[CSC/ECE 517 Spring 2016/Refactor different question types from quiz feature]]

*[[CSC/ECE 517 Spring 2016/Implement private browsing]]

*[[CSC/ECE 517 Spring 2016/Write automated tests for WebDriver]]

*[[CSC/ECE 517 Spring 2016/Refactor sign_up_sheet_controller.rb and sign_up_topic.rb]]

*[[CSC/ECE 517 Spring 2016/Implement Common Parts of the CSSOM API]]

*[[CSC/ECE 517 Spring 2016/Functional tests for assignment creation function]]

*[[CSC/ECE 517 Spring 2016/E1604. Functional tests for Calibration function]]

*[[CSC/ECE 517 Spring 2016/Refactor and write unit tests for question type.rb]]

*[[CSC/ECE 517 Spring 2016/Refactor response controller]]

*[[CSC/ECE_517_Spring_2016_OSS_M1606]]

*[[CSC/ECE_517_Spring_2016/OSS_E1601]]

==Final Project Design Document==
*[[CSC/ECE 517 Spring 2016 M1601 Implement HTML5 form validation]]
*[[CSC/ECE 517 Spring 2016 E1624 Compose functional tests for Suggest topics of student functionality ]]
*[[CSC/ECE 517 Spring 2016 E1631 Team-based reviewing]]
*[[CSC/ECE 517 Spring 2016 E1628 Visualization]]
*[[CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers]]

/CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:22:53Z

Bghagar: moved /CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers to CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers: Remove initial backslash

#REDIRECT [[CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers]]

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:22:53Z

==Show confidence ratings for grade, based on reputations of reviewers==

===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===Motivation===

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

===Description of confidence rating and our system===

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

===Section: Design Pattern - Model-view-controller(MVC)===

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

===Section: Documentation===

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Section: Classes Involved===

===Section: Files Involved===

===Models:===

participant

===Views:===

grades/view.html.erb

grades/view_my_scores.html.erb

===Controllers:===

Reputation_web_service_controller.rb

===Standards:===

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

===Section: Tasks Performed:===

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Section: Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===Section: References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

User:Bghagar

2016-04-04T02:22:17Z

Bghagar: moved User:Bghagar to /CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers: Migrate to main page

#REDIRECT [[/CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers]]

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-04T02:22:17Z

Bghagar: moved User:Bghagar to /CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers: Migrate to main page

==Show confidence ratings for grade, based on reputations of reviewers==

===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===Motivation===

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

===Description of confidence rating and our system===

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

===Section: Design Pattern - Model-view-controller(MVC)===

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

===Section: Documentation===

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Section: Classes Involved===

===Section: Files Involved===

===Models:===

participant

===Views:===

grades/view.html.erb

grades/view_my_scores.html.erb

===Controllers:===

Reputation_web_service_controller.rb

===Standards:===

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

===Section: Tasks Performed:===

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Section: Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===Section: References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-03T09:11:36Z

Bghagar:

==Show confidence ratings for grade, based on reputations of reviewers==

===Introduction to Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

===Motivation===

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

===Description of confidence rating and our system===

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

===Section: Design Pattern - Model-view-controller(MVC)===

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

===Section: Documentation===

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

===Section: Classes Involved===

===Section: Files Involved===

===Models:===

participant

===Views:===

grades/view.html.erb

grades/view_my_scores.html.erb

===Controllers:===

Reputation_web_service_controller.rb

===Standards:===

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

===Section: Tasks Performed:===

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

===Section: Testing===

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

===Section: References===

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-03T09:08:31Z

Bghagar:

'''Show confidence ratings for grade, based on reputations of reviewers'''

'''Introduction to Expertiza'''

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

'''Motivation'''

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

'''Description of confidence rating and our system'''

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

'''Section: Design Pattern - Model-view-controller(MVC)'''

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

'''Section: Documentation'''

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

'''Section: Classes Involved'''

'''Section: Files Involved'''

'''Models:'''

participant

'''Views:'''

grades/view.html.erb

grades/view_my_scores.html.erb

'''Controllers:'''

Reputation_web_service_controller.rb

'''Standards:'''

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

'''Section: Tasks Performed:'''

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

'''Section: Testing'''

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

'''Section: References'''

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-03T08:57:57Z

Bghagar:

'''Show confidence ratings for grade, based on reputations of reviewers'''

'''Introduction to Expertiza'''

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

'''Motivation'''

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

'''Description of confidence rating and our system'''

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

'''Section: Design Pattern - Model-view-controller(MVC)'''

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

'''Section: Documentation'''

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

'''Section: Classes Involved'''

'''Section: Files Involved'''

'''Models:'''

participant

'''Views:'''

grades/view.html.erb

grades/view_my_scores.html.erb

'''Controllers:'''

Reputation_web_service_controller.rb

'''Standards:'''

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

'''Section: Tasks Performed:'''

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

'''Section: Testing'''

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

'''Section: References'''

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

CSC/ECE 517 Spring 2016 E1625 Show confidence ratings for grade based on reputations of reviewers

2016-04-03T08:44:11Z

Bghagar: Created page with "'''Show confidence ratings for grade, based on reputations of reviewers''' '''Introduction to Expertiza''' [http://expertiza.ncsu.edu/ Expertiza] is a web application develo..."

'''Show confidence ratings for grade, based on reputations of reviewers'''

'''Introduction to Expertiza'''

[http://expertiza.ncsu.edu/ Expertiza] is a web application developed using the [http://rubyonrails.org/ Ruby on Rails] framework as a combined effort of students and faculty. The main advantage of using Expertiza, in an educational environment, is for the instructor to introduce peer reviewing among the students. The instructors can create and customize assignments, create a list of topics the students can sign up for, have students work on teams and then review each other's assignments at the end. Expertiza supports submission of almost any document type, including the URLs and wiki pages.

Expertiza is supported by National Science Foundation under Grant No. 0536558. Additional funding from the [http://www.ncsu.edu/ NCSU] [http://litre.ncsu.edu/ Learning in a Technology-Rich Environment] (LITRE) program, the NCSU [http://ofd.ncsu.edu/teaching-learning/ Faculty Center for Teaching and Learning], the NCSU [http://stem.ncsu.edu/ STEM] Initiative, and the Center for Advanced Computing and Communication

'''Motivation'''

Students select from a list of tasks to be performed, then they prepare their work and submit it to a peer-review system. The work is then reviewed by other students, who offer comments/feedback to help the submitters improve their work.

During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.

Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.

'''Description of confidence rating and our system'''

[https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail Hamer’s and Lauw’s algorithm]

There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”

The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. Additionally the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].

'''Section: Design Pattern - Model-view-controller(MVC)'''

In an MVC pattern, a software application is divided into three major components : Model, View and Controller. This allows separation of internal representations of informations from the information that needs to be presented to the user.

Model - Maintains the data, logic and the relationships between objects and the database. It also handles validations, associations and transactions.

View - Displays data received from the controller in the User Interface.

Controller - Accepts input from client and converts it into actionable items for the model or the view. Its responsibilities include querying models and organizing data into a structure that is displayed by the view

'''Section: Documentation'''

All the documentation for the Expertiza system can be found in the following links: [https://github.com/expertiza/expertiza Expertiza on Github] , [http://wikis.lib.ncsu.edu/index.php/Expertiza Wiki documentation for Expertiza]

'''Section: Classes Involved'''

'''Section: Files Involved'''

'''Models:'''

participant

'''Views:'''

grades/view.html.erb

grades/view_my_scores.html.erb

'''Controllers:'''

Reputation_web_service_controller.rb

'''Standards:'''

All our code follows the [https://docs.google.com/document/d/1qQD7fcypFk77nq7Jx7ZNyCNpLyt1oXKaq5G-W7zkV3k/edit global rules] on Ruby style.

'''Section: Tasks Performed:'''

● Write a migration to add reputation field in participant table. (The default value of this field is 1, set any nil to 1)

● Change the send_post_request method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to obtain initial reputation values from DB instead of hard-coded.

● Write a new method in [https://github.com/expertiza/expertiza/blob/master/app/controllers/reputation_web_service_controller.rb reputation_web_service_controller] to extract reputation values from JSON response and insert into DB.

● On view_my_score (student-end) and view_scores (instructor-end) pages: use different font color on students’ names to distinguish different level of reputation values

○ Reputation range of Hamer’s algorithm is :

red value < 0.5

yellow value is > 0.5 to 1

orange value is > 1 to 1.5

light green value is > 1.5 to 2

green value is > 2

○ Reputation range of Lauw’s algorithm is

red value is < 0.2

yellow value is < 0.2 to 0.4

orange value is > 0.4 to 0.6

light green value is > 0.6 to 0.8

green value is > 0.8.

● Instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

'''Section: Testing'''

We will be using the existing test suite used by gem to test any new code modification. We will be writing new test cases for any new public method exposed by existing classes.

'''Section: References'''

'''1. [https://github.com/expertiza/expertiza Expertiza on GitHub (Original version)]'''

'''2. [http://expertiza.ncsu.edu/ Expertiza]'''

'''Previous Design documents''' http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2015

CSC/ECE 517 Spring 2016/Functional tests for assignment creation function

2016-03-26T06:30:20Z

Bghagar:

=Functional tests for assignment creation function=
This page outlines the use tests for assignment creation using the Expertiza system
__TOC__

===Structural Overview===
Assignment creation in Expertiza provides a large number of options for users. Features come at the cost of complexity; this project is focused on creating testing methods to ensure that user interaction with the assignment interface remains stable and reliable.

Assignment creation is available from the Manage -> Assignments drop down on the expertiza site. Once an assignment is created it has 5 tabs for form entry. The remainder of this document will cover testing for each tab in more detail, the tabs are:
* General
* Topics
* Rubrics
* Review Strategy

Check the panxing01/expertiza repository for the current development files. Currently all tests are run in two files:
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_create_spec.rb
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_interface_spec.rb

===Setup===
To setup functionality testing of assignment creation feature we did the following steps:
*Created RSpec file in /spec/features/ folder of [https://github.com/expertiza/expertiza Expertiza]
* Wrote functional tests using Capybara

===General===
In this section, the functional test includes assignment name, course, submission directory, description URL, "Has team?", "Has quiz?" and "Calibrated peer-review for training".
*assignment name
<pre>
it "can set assginment name" do
login_as("instructor6")
visit '/assignments/new?private=0'
expect(page).to have_content "Assignment name"
fill_in "Assignment name:", with: 'assignment for test'
click_button "Create"
expect(Assignment.where(name: "assignment for test")).to exist
end
</pre>
*course
<pre>
it "can select course" do
login_as("instructor6")
visit '/assignments/new?private=0'
select('5', :from => 'assignment_form_assignment_course_id')
click_button "Create"
end
</pre>
*submission directory
<pre>
it "can set assginment Submission directory" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 2'
expect(page).to have_content "Submission directory"
fill_in "Submission directory:", with: 'Submission directory for test'
click_button "Create"
expect(Assignment.find(2).directory_path).to eq 'Submission directory for test'
end
</pre>
*description URL
<pre>
it "can set assginment Description URL" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 3'
expect(page).to have_content "Description URL"
fill_in "Description URL:", with: 'Description URL for test'
click_button "Create"
expect(Assignment.find(2).spec_location).to eq 'Description URL for test'
end
</pre>
*"Has team?"
<pre>
it "can Has teams and set team number" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 5'
find('#team_assignment').set(true)
expect(page).to have_content "Maximum number of members per team"
expect(page).to have_content "Show teammate reviews"
fill_in 'assignment_form_assignment_max_team_size', with: 4
find('#assignment_form_assignment_show_teammate_reviews').set(true)
click_button "Create"
expect(Assignment.find(2).show_teammate_reviews).to eq true
end
</pre>
*"Has quiz?"
<pre>
it "can save the number of quiz questions" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: '11'
select('5', :from => 'assignment_form_assignment_course_id')
find('#assignment_form_assignment_require_quiz').set(true)
click_button "Create"
visit '/assignments/2/edit'
expect(page).to have_content "Number of Quiz questions"
end
</pre>
*"Calibrated peer-review for training"
<pre>
it "can set Calibrated peer-view for training" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 4'
find('#assignment_form_assignment_is_calibrated').set(true)
click_button "Create"
expect(Assignment.find(2).is_calibrated).to eq true
end
</pre>
===Topics===
In this section, functionality of "Edit topics" is thoroughly tested. Below are some of the code snippets of this testing module:
*Edit topics content
<pre>
it "is able to see edit topics content correctly" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'
expect(page).to have_content("Allow topic suggestions from students?")
expect(page).to have_content("Enable bidding for topics?")
expect(page).to have_content("Enable authors to review others working on same topic?")
expect(page).to have_content("Allow reviewer to choose which topic to review?")
expect(page).to have_content("Allow participants to create bookmarks? ")
expect(page).to have_content("Hide all teams")
expect(page).to have_content("New topic")
expect(page).to have_content("Import Topics")
expect(page).to have_content("Back")
expect(page).to have_content("Save")
end
</pre>

*Show/hide teams
<pre>
it "is able to show/hide teams" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'

click_link('Hide all teams')

expect(page).to have_content("Show all teams")

click_link("Show all teams")
expect(page).to have_content("Hide all teams")
end
</pre>

===Rubrics===
Rubrics are arranged into 3 rows of data named "Review", "Author Feedback", and "Teammate Review". Each row is handled under separate descriptions in order to allow quick and comprehensive testing for the seperate functions.
Each row updates using a subclass of the Questionnaire class for dropdown values. Classes used are the ReviewQuestionnaire, AuthorFeedbackQuestionnaire, and the TeammateReviewQuestionnaire respectively.

Testing required creation of the three questionnaire subclasses in order to populate the dropdowns. Basic code layout for testing each value change:
<pre>
describe "Edit [subclass] rubric" do
it "should update [component]", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("[table row identifier]") do
select "[subclass instance name value]", :from => '[form identifier]'
end
click_button 'Save'
expect([questionnaireinstance]).to have_attributes([attrib updated] => [new value])
end
</pre>
===Review Strategy===
In this module we have tested the functionality of the "Teammate Review" module. Some of the tests are given below:
*Teammate Review
<pre>
it "should update questionnaire", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2")).to exist
end
</pre>

*Notification limit
<pre>
it "should update notification limit", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', :with => '50'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2").first).to have_attributes(:notification_limit => 50)
end
</pre>

===Class Hierarchy===
The Assignment class has many questionnaires throught the AssignmentQuestionnairs class. The Questionnaire class inherits from ActiveRecord::Base and has 8 fields plus primary key and date entries. The fields are:
* name: The name of the questionnaire row
* instructor_id: The instuctor key linked to the row
* private: True if private questionnaire
* min_question_score
* max_question_score
* type: field showing subclass
* display_type
* instruction_loc: stores URL for questionnaire instruction

The Questionnaire class has 11 sub-classes. They are:
* Rubric
* CourseEvaluationQuestionnaire
* SurveyQuestionnaire
* BookmarkRatingQuestionnaire
* AuthorFeedbackQuestionnaire
* QuizQuestionnaire
* GlobalSurveyQuestionnaire
* Metasurvey
* ReviewQuestionnaire
* MetareviewQuestionnaire
* TeammateReviewQuestionnaire

CSC/ECE 517 Spring 2016/Functional tests for assignment creation function

2016-03-26T06:29:33Z

Bghagar:

=Functional tests for assignment creation function=
This page outlines the use tests for assignment creation using the Expertiza system
__TOC__

===Structural Overview===
Assignment creation in Expertiza provides a large number of options for users. Features come at the cost of complexity; this project is focused on creating testing methods to ensure that user interaction with the assignment interface remains stable and reliable.

Assignment creation is available from the Manage -> Assignments drop down on the expertiza site. Once an assignment is created it has 5 tabs for form entry. The remainder of this document will cover testing for each tab in more detail, the tabs are:
* General
* Topics
* Rubrics
* Review Strategy

Check the panxing01/expertiza repository for the current development files. Currently all tests are run in two files:
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_create_spec.rb
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_interface_spec.rb

===Setup===
To setup functionality testing of assignment creation feature we did the following steps:
*Created RSpec file in /spec/features/ folder of [https://github.com/expertiza/expertiza Expertiza]
* Wrote functional tests using Capybara

===General===
In this section, the functional test includes assignment name, course, submission directory, description URL, "Has team?", "Has quiz?" and "Calibrated peer-review for training".
*assignment name
<pre>
it "can set assginment name" do
login_as("instructor6")
visit '/assignments/new?private=0'
expect(page).to have_content "Assignment name"
fill_in "Assignment name:", with: 'assignment for test'
click_button "Create"
expect(Assignment.where(name: "assignment for test")).to exist
end
</pre>
*course
<pre>
it "can select course" do
login_as("instructor6")
visit '/assignments/new?private=0'
select('5', :from => 'assignment_form_assignment_course_id')
click_button "Create"
end
</pre>
*submission directory
<pre>
it "can set assginment Submission directory" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 2'
expect(page).to have_content "Submission directory"
fill_in "Submission directory:", with: 'Submission directory for test'
click_button "Create"
expect(Assignment.find(2).directory_path).to eq 'Submission directory for test'
end
</pre>
*description URL
<pre>
it "can set assginment Description URL" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 3'
expect(page).to have_content "Description URL"
fill_in "Description URL:", with: 'Description URL for test'
click_button "Create"
expect(Assignment.find(2).spec_location).to eq 'Description URL for test'
end
</pre>
*"Has team?"
<pre>
it "can Has teams and set team number" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 5'
find('#team_assignment').set(true)
expect(page).to have_content "Maximum number of members per team"
expect(page).to have_content "Show teammate reviews"
fill_in 'assignment_form_assignment_max_team_size', with: 4
find('#assignment_form_assignment_show_teammate_reviews').set(true)
click_button "Create"
expect(Assignment.find(2).show_teammate_reviews).to eq true
end
</pre>
*"Has quiz?"
<pre>
it "can save the number of quiz questions" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: '11'
select('5', :from => 'assignment_form_assignment_course_id')
find('#assignment_form_assignment_require_quiz').set(true)
click_button "Create"
visit '/assignments/2/edit'
expect(page).to have_content "Number of Quiz questions"
end
</pre>
*"Calibrated peer-review for training"
<pre>
it "can set Calibrated peer-view for training" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 4'
find('#assignment_form_assignment_is_calibrated').set(true)
click_button "Create"
expect(Assignment.find(2).is_calibrated).to eq true
end
</pre>
===Topics===
In this section, functionality of "Edit topics" is thoroughly tested. Below are some of the code snippets of this testing module:
*Edit topics content
<pre>
it "is able to see edit topics content correctly" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'
expect(page).to have_content("Allow topic suggestions from students?")
expect(page).to have_content("Enable bidding for topics?")
expect(page).to have_content("Enable authors to review others working on same topic?")
expect(page).to have_content("Allow reviewer to choose which topic to review?")
expect(page).to have_content("Allow participants to create bookmarks? ")
expect(page).to have_content("Hide all teams")
expect(page).to have_content("New topic")
expect(page).to have_content("Import Topics")
expect(page).to have_content("Back")
expect(page).to have_content("Save")
end
</pre>

*Show/hide teams
<pre>
it "is able to show/hide teams" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'

click_link('Hide all teams')

expect(page).to have_content("Show all teams")

click_link("Show all teams")
expect(page).to have_content("Hide all teams")
end
</pre>

===Rubrics===
Rubrics are arranged into 3 rows of data named "Review", "Author Feedback", and "Teammate Review". Each row is handled under separate descriptions in order to allow quick and comprehensive testing for the seperate functions.
Each row updates using a subclass of the Questionnaire class for dropdown values. Classes used are the ReviewQuestionnaire, AuthorFeedbackQuestionnaire, and the TeammateReviewQuestionnaire respectively. The Active Record hierarchy is:

Testing required creation of the three questionnaire subclasses in order to populate the dropdowns. Basic code layout for testing each value change:
<pre>
describe "Edit [subclass] rubric" do
it "should update [component]", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("[table row identifier]") do
select "[subclass instance name value]", :from => '[form identifier]'
end
click_button 'Save'
expect([questionnaireinstance]).to have_attributes([attrib updated] => [new value])
end
</pre>
===Review Strategy===
In this module we have tested the functionality of the "Teammate Review" module. Some of the tests are given below:
*Teammate Review
<pre>
it "should update questionnaire", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2")).to exist
end
</pre>

*Notification limit
<pre>
it "should update notification limit", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', :with => '50'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2").first).to have_attributes(:notification_limit => 50)
end
</pre>

===Class Hierarchy===
The Assignment class has many questionnaires throught the AssignmentQuestionnairs class. The Questionnaire class inherits from ActiveRecord::Base and has 8 fields plus primary key and date entries. The fields are:
* name: The name of the questionnaire row
* instructor_id: The instuctor key linked to the row
* private: True if private questionnaire
* min_question_score
* max_question_score
* type: field showing subclass
* display_type
* instruction_loc: stores URL for questionnaire instruction

The Questionnaire class has 11 sub-classes. They are:
* Rubric
* CourseEvaluationQuestionnaire
* SurveyQuestionnaire
* BookmarkRatingQuestionnaire
* AuthorFeedbackQuestionnaire
* QuizQuestionnaire
* GlobalSurveyQuestionnaire
* Metasurvey
* ReviewQuestionnaire
* MetareviewQuestionnaire
* TeammateReviewQuestionnaire

CSC/ECE 517 Spring 2016/Functional tests for assignment creation function

2016-03-26T06:10:44Z

Bghagar:

=Functional tests for assignment creation function=
This page outlines the use tests for assignment creation using the Expertiza system
__TOC__

===Structural Overview===
Assignment creation in Expertiza provides a large number of options for users. Features come at the cost of complexity; this project is focused on creating testing methods to ensure that user interaction with the assignment interface remains stable and reliable.

Assignment creation is available from the Manage -> Assignments drop down on the expertiza site. Once an assignment is created it has 5 tabs for form entry. The remainder of this document will cover testing for each tab in more detail, the tabs are:
* General
* Topics
* Rubrics
* Review Strategy

Check the panxing01/expertiza repository for the current development files. Currently all tests are run in two files:
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_create_spec.rb
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_interface_spec.rb

===Setup===
To setup functionality testing of assignment creation feature we did the following steps:
*Created RSpec file in /spec/features/ folder of [https://github.com/expertiza/expertiza Expertiza]
* Wrote functional tests using Capybara

===General===
In this section, the functional test includes assignment name, course, submission directory, description URL, "Has team?", "Has quiz?" and "Calibrated peer-review for training".
*assignment name
<pre>
it "can set assginment name" do
login_as("instructor6")
visit '/assignments/new?private=0'
expect(page).to have_content "Assignment name"
fill_in "Assignment name:", with: 'assignment for test'
click_button "Create"
expect(Assignment.where(name: "assignment for test")).to exist
end
</pre>
*course
<pre>
it "can select course" do
login_as("instructor6")
visit '/assignments/new?private=0'
select('5', :from => 'assignment_form_assignment_course_id')
click_button "Create"
end
</pre>
*submission directory
<pre>
it "can set assginment Submission directory" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 2'
expect(page).to have_content "Submission directory"
fill_in "Submission directory:", with: 'Submission directory for test'
click_button "Create"
expect(Assignment.find(2).directory_path).to eq 'Submission directory for test'
end
</pre>
*description URL
<pre>
it "can set assginment Description URL" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 3'
expect(page).to have_content "Description URL"
fill_in "Description URL:", with: 'Description URL for test'
click_button "Create"
expect(Assignment.find(2).spec_location).to eq 'Description URL for test'
end
</pre>
*"Has team?"
<pre>
it "can Has teams and set team number" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 5'
find('#team_assignment').set(true)
expect(page).to have_content "Maximum number of members per team"
expect(page).to have_content "Show teammate reviews"
fill_in 'assignment_form_assignment_max_team_size', with: 4
find('#assignment_form_assignment_show_teammate_reviews').set(true)
click_button "Create"
expect(Assignment.find(2).show_teammate_reviews).to eq true
end
</pre>
*"Has quiz?"
<pre>
it "can save the number of quiz questions" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: '11'
select('5', :from => 'assignment_form_assignment_course_id')
find('#assignment_form_assignment_require_quiz').set(true)
click_button "Create"
visit '/assignments/2/edit'
expect(page).to have_content "Number of Quiz questions"
end
</pre>
*"Calibrated peer-review for training"
<pre>
it "can set Calibrated peer-view for training" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 4'
find('#assignment_form_assignment_is_calibrated').set(true)
click_button "Create"
expect(Assignment.find(2).is_calibrated).to eq true
end
</pre>
===Topics===
In this section, functionality of "Edit topics" is thoroughly tested. Below are some of the code snippets of this testing module:
*Edit topics content
<pre>
it "is able to see edit topics content correctly" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'
expect(page).to have_content("Allow topic suggestions from students?")
expect(page).to have_content("Enable bidding for topics?")
expect(page).to have_content("Enable authors to review others working on same topic?")
expect(page).to have_content("Allow reviewer to choose which topic to review?")
expect(page).to have_content("Allow participants to create bookmarks? ")
expect(page).to have_content("Hide all teams")
expect(page).to have_content("New topic")
expect(page).to have_content("Import Topics")
expect(page).to have_content("Back")
expect(page).to have_content("Save")
end
</pre>

*Show/hide teams
<pre>
it "is able to show/hide teams" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'

click_link('Hide all teams')

expect(page).to have_content("Show all teams")

click_link("Show all teams")
expect(page).to have_content("Hide all teams")
end
</pre>

===Rubrics===
Rubrics are arranged into 3 rows of data named "Review", "Author Feedback", and "Teammate Review". Each row is handled under separate descriptions in order to allow quick and comprehensive testing for the seperate functions.
Each row updates a subclass of the Questionnaire class. The Active Record hierarchy is:
* Questionnaire
** ReviewQuestionnaire
** AuthorFeedbackQuestionnaire
** TeammateReviewQuestionaire

The Questionnaire class is connected to the Assignment class via the AssignmentQuestionnaire class.

===Review Strategy===
In this module we have tested the functionality of the "Teammate Review" module. Some of the tests are given below:
*Teammate Review
<pre>
it "should update questionnaire", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2")).to exist
end
</pre>

*Notification limit
<pre>
it "should update notification limit", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', :with => '50'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2").first).to have_attributes(:notification_limit => 50)
end
</pre>

===Class Hierarchy===
The Questionnaire class inherits from ActiveRecord::Base and has 8 fields plus primary key and date entries. The fields are:
* name: The name of the questionnaire row
* instructor_id: The instuctor key linked to the row
* private: True if private questionnaire
* min_question_score
* max_question_score
* type: field showing subclass
* display_type
* instruction_loc: stores URL for questionnaire instruction

The Questionnaire class has 11 sub-classes. They are:
* Rubric
* CourseEvaluationQuestionnaire
* SurveyQuestionnaire
* BookmarkRatingQuestionnaire
* AuthorFeedbackQuestionnaire
* QuizQuestionnaire
* GlobalSurveyQuestionnaire
* Metasurvey
* ReviewQuestionnaire
* MetareviewQuestionnaire
* TeammateReviewQuestionnaire

CSC/ECE 517 Spring 2016/Functional tests for assignment creation function

2016-03-26T06:09:44Z

Bghagar:

=Functional tests for assignment creation function=
This page outlines the use tests for assignment creation using the Expertiza system
__TOC__

===Structural Overview===
Assignment creation in Expertiza provides a large number of options for users. Features come at the cost of complexity; this project is focused on creating testing methods to ensure that user interaction with the assignment interface remains stable and reliable.

Assignment creation is available from the Manage -> Assignments drop down on the expertiza site. Once an assignment is created it has 5 tabs for form entry. The remainder of this document will cover testing for each tab in more detail, the tabs are:
* General
* Topics
* Rubrics
* Review Strategy

Check the panxing01/expertiza repository for the current development files. Currently all tests are run in two files:
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_create_spec.rb
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_interface_spec.rb

===Setup===
To setup functionality testing of assignment creation feature we did the following steps:
*Created RSpec file in /spec/features/ folder of [https://github.com/expertiza/expertiza Expertiza]
* Wrote functional tests using Capybara

===General===
In this section, the functional test includes assignment name, course, submission directory, description URL, "Has team?", "Has quiz?" and "Calibrated peer-review for training".
*assignment name
<pre>
it "can set assginment name" do
login_as("instructor6")
visit '/assignments/new?private=0'
expect(page).to have_content "Assignment name"
fill_in "Assignment name:", with: 'assignment for test'
click_button "Create"
expect(Assignment.where(name: "assignment for test")).to exist
end
</pre>
*course
<pre>
it "can select course" do
login_as("instructor6")
visit '/assignments/new?private=0'
select('5', :from => 'assignment_form_assignment_course_id')
click_button "Create"
end
</pre>
*submission directory
<pre>
it "can set assginment Submission directory" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 2'
expect(page).to have_content "Submission directory"
fill_in "Submission directory:", with: 'Submission directory for test'
click_button "Create"
expect(Assignment.find(2).directory_path).to eq 'Submission directory for test'
end
</pre>
*description URL
<pre>
it "can set assginment Description URL" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 3'
expect(page).to have_content "Description URL"
fill_in "Description URL:", with: 'Description URL for test'
click_button "Create"
expect(Assignment.find(2).spec_location).to eq 'Description URL for test'
end
</pre>
*"Has team?"
<pre>
it "can Has teams and set team number" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 5'
find('#team_assignment').set(true)
expect(page).to have_content "Maximum number of members per team"
expect(page).to have_content "Show teammate reviews"
fill_in 'assignment_form_assignment_max_team_size', with: 4
find('#assignment_form_assignment_show_teammate_reviews').set(true)
click_button "Create"
expect(Assignment.find(2).show_teammate_reviews).to eq true
end
</pre>
*"Has quiz?"
<pre>
it "can save the number of quiz questions" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: '11'
select('5', :from => 'assignment_form_assignment_course_id')
find('#assignment_form_assignment_require_quiz').set(true)
click_button "Create"
visit '/assignments/2/edit'
expect(page).to have_content "Number of Quiz questions"
end
</pre>
*"Calibrated peer-review for training"
<pre>
it "can set Calibrated peer-view for training" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 4'
find('#assignment_form_assignment_is_calibrated').set(true)
click_button "Create"
expect(Assignment.find(2).is_calibrated).to eq true
end
</pre>
===Topics===
In this section, functionality of "Edit topics" is thoroughly tested. Below are some of the code snippets of this testing module:
*Edit topics content
<pre>
it "is able to see edit topics content correctly" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'
expect(page).to have_content("Allow topic suggestions from students?")
expect(page).to have_content("Enable bidding for topics?")
expect(page).to have_content("Enable authors to review others working on same topic?")
expect(page).to have_content("Allow reviewer to choose which topic to review?")
expect(page).to have_content("Allow participants to create bookmarks? ")
expect(page).to have_content("Hide all teams")
expect(page).to have_content("New topic")
expect(page).to have_content("Import Topics")
expect(page).to have_content("Back")
expect(page).to have_content("Save")
end
</pre>

*Show/hide teams
<pre>
it "is able to show/hide teams" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'

click_link('Hide all teams')

expect(page).to have_content("Show all teams")

click_link("Show all teams")
expect(page).to have_content("Hide all teams")
end
</pre>

===Rubrics===
====Class Hierarchy====
The Questionnaire class inherits from ActiveRecord::Base and has 8 fields plus primary key and date entries. The fields are:
* name: The name of the questionnaire row
* instructor_id: The instuctor key linked to the row
* private: True if private questionnaire
* min_question_score
* max_question_score
* type: field showing subclass
* display_type
* instruction_loc: stores URL for questionnaire instruction

The Questionnaire class has 11 sub-classes. They are:
* Rubric
* CourseEvaluationQuestionnaire
* SurveyQuestionnaire
* BookmarkRatingQuestionnaire
* AuthorFeedbackQuestionnaire
* QuizQuestionnaire
* GlobalSurveyQuestionnaire
* Metasurvey
* ReviewQuestionnaire
* MetareviewQuestionnaire
* TeammateReviewQuestionnaire

Rubrics are arranged into 3 rows of data named "Review", "Author Feedback", and "Teammate Review". Each row is handled under separate descriptions in order to allow quick and comprehensive testing for the seperate functions.
Each row updates a subclass of the Questionnaire class. The Active Record hierarchy is:
* Questionnaire
** ReviewQuestionnaire
** AuthorFeedbackQuestionnaire
** TeammateReviewQuestionaire

The Questionnaire class is connected to the Assignment class via the AssignmentQuestionnaire class.

===Review Strategy===
In this module we have tested the functionality of the "Teammate Review" module. Some of the tests are given below:
*Teammate Review
<pre>
it "should update questionnaire", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2")).to exist
end
</pre>

*Notification limit
<pre>
it "should update notification limit", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', :with => '50'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2").first).to have_attributes(:notification_limit => 50)
end
</pre>

CSC/ECE 517 Spring 2016/Functional tests for assignment creation function

2016-03-25T16:32:49Z

Bghagar:

=Functional tests for assignment creation function=
This page outlines the use tests for assignment creation using the Expertiza system
__TOC__

===Structural Overview===
Assignment creation in Expertiza provides a large number of options for users. Features come at the cost of complexity; this project is focused on creating testing methods to ensure that user interaction with the assignment interface remains stable and reliable.

Assignment creation is available from the Manage -> Assignments drop down on the expertiza site. Once an assignment is created it has 5 tabs for form entry. The remainder of this document will cover testing for each tab in more detail, the tabs are:
* General
* Topics
* Rubrics
* Review Strategy

Check the panxing01/expertiza repository for the current development files. Currently all tests are run in two files:
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_create_spec.rb
* https://github.com/panxing01/expertiza/blob/master/spec/features/assignment_interface_spec.rb

===Setup===
To setup functionality testing of assignment creation feature we did the following steps:
*Created RSpec file in /spec/features/ folder of [https://github.com/expertiza/expertiza Expertiza]
* Wrote functional tests using Capybara

===General===
In this section, the functional test includes assignment name, course, submission directory, description URL, "Has team?", "Has quiz?" and "Calibrated peer-review for training".
*assignment name
<pre>
it "can set assginment name" do
login_as("instructor6")
visit '/assignments/new?private=0'
expect(page).to have_content "Assignment name"
fill_in "Assignment name:", with: 'assignment for test'
click_button "Create"
expect(Assignment.where(name: "assignment for test")).to exist
end
</pre>
*course
<pre>
it "can select course" do
login_as("instructor6")
visit '/assignments/new?private=0'
select('5', :from => 'assignment_form_assignment_course_id')
click_button "Create"
end
</pre>
*submission directory
<pre>
it "can set assginment Submission directory" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 2'
expect(page).to have_content "Submission directory"
fill_in "Submission directory:", with: 'Submission directory for test'
click_button "Create"
expect(Assignment.find(2).directory_path).to eq 'Submission directory for test'
end
</pre>
*description URL
<pre>
it "can set assginment Description URL" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 3'
expect(page).to have_content "Description URL"
fill_in "Description URL:", with: 'Description URL for test'
click_button "Create"
expect(Assignment.find(2).spec_location).to eq 'Description URL for test'
end
</pre>
*"Has team?"
<pre>
it "can Has teams and set team number" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 5'
find('#team_assignment').set(true)
expect(page).to have_content "Maximum number of members per team"
expect(page).to have_content "Show teammate reviews"
fill_in 'assignment_form_assignment_max_team_size', with: 4
find('#assignment_form_assignment_show_teammate_reviews').set(true)
click_button "Create"
expect(Assignment.find(2).show_teammate_reviews).to eq true
end
</pre>
*"Has quiz?"
<pre>
it "can save the number of quiz questions" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: '11'
select('5', :from => 'assignment_form_assignment_course_id')
find('#assignment_form_assignment_require_quiz').set(true)
click_button "Create"
visit '/assignments/2/edit'
expect(page).to have_content "Number of Quiz questions"
end
</pre>
*"Calibrated peer-review for training"
<pre>
it "can set Calibrated peer-view for training" do
login_as("instructor6")
visit '/assignments/new?private=0'
fill_in "Assignment name:", with: 'assignment for test 4'
find('#assignment_form_assignment_is_calibrated').set(true)
click_button "Create"
expect(Assignment.find(2).is_calibrated).to eq true
end
</pre>
===Topics===
In this section, functionality of "Edit topics" is thoroughly tested. Below are some of the code snippets of this testing module:
*Edit topics content
<pre>
it "is able to see edit topics content correctly" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'
expect(page).to have_content("Allow topic suggestions from students?")
expect(page).to have_content("Enable bidding for topics?")
expect(page).to have_content("Enable authors to review others working on same topic?")
expect(page).to have_content("Allow reviewer to choose which topic to review?")
expect(page).to have_content("Allow participants to create bookmarks? ")
expect(page).to have_content("Hide all teams")
expect(page).to have_content("New topic")
expect(page).to have_content("Import Topics")
expect(page).to have_content("Back")
expect(page).to have_content("Save")
end
</pre>

*Show/hide teams
<pre>
it "is able to show/hide teams" do
login_as("instructor6")
visit 'assignments/1/edit#tab-2'

click_link('Hide all teams')

expect(page).to have_content("Show all teams")

click_link("Show all teams")
expect(page).to have_content("Hide all teams")
end
</pre>

===Rubrics===
Rubrics are arranged into 3 rows of data named "Review", "Author Feedback", and "Teammate Review". Each row is handled under separate descriptions in order to allow quick and comprehensive testing for the seperate functions.
Each row updates a subclass of the Questionnaire class. The Active Record hierarchy is:
* Questionnaire
** ReviewQuestionnaire
** AuthorFeedbackQuestionnaire
** TeammateReviewQuestionaire

The Questionnaire class is connected to the Assignment class via the AssignmentQuestionnaire class.

===Review Strategy===
In this module we have tested the functionality of the "Teammate Review" module. Some of the tests are given below:
*Teammate Review
<pre>
it "should update questionnaire", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2")).to exist
end
</pre>

*Notification limit
<pre>
it "should update notification limit", js: true do
login_as("instructor6")
visit '/assignments/1/edit'
find_link('Rubrics').click
within("tr#questionnaire_table_TeammateReviewQuestionnaire") do
select "TeammateReviewQuestionnaire2", :from => 'assignment_form[assignment_questionnaire][][questionnaire_id]'
fill_in 'assignment_form[assignment_questionnaire][][notification_limit]', :with => '50'
end
click_button 'Save'
expect(get_questionnaire("TeammateReviewQuestionnaire2").first).to have_attributes(:notification_limit => 50)
end
</pre>

User:Xpan

2016-03-24T07:37:32Z

Bghagar: small updates, needs a lot more work

User:Xpan

2016-03-24T07:23:10Z

Bghagar: Formatting, added intro