Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:52:34Z

Npatil2: /* Links */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
* Single Responsibility Principle (SRP):
Each method within the QuestionnaireHelper class appears to adhere to the SRP by focusing on a single task or responsibility. For example, the adjust_advice_size method is responsible for adjusting the size of advice based on questionnaire scores, while the questionnaire_factory method is responsible for creating instances of questionnaire types based on the given type parameter. This adherence ensures that each method has a clear and distinct purpose, promoting code maintainability and readability.

* Open/Closed Principle (OCP):
While not explicitly evident in the provided snippets, the design allows for extension without modification, which aligns with the OCP. For instance, new types of questionnaires can be added without altering existing code by simply extending the questionnaire_factory method to accommodate the new types.

* Dependency Injection Principle (DIP):
The methods in the QuestionnaireHelper class accept various objects (e.g., questionnaire, scored_question) as parameters, adhering to the principle of dependency injection. By accepting dependencies from external sources rather than creating them internally, these methods become more flexible and easily testable.

* Factory Method Pattern:
The questionnaire_factory method can be seen as exhibiting characteristics of the factory method pattern. It dynamically creates instances of different questionnaire types based on the given type parameter, promoting flexibility and extensibility.

After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
* Single Responsibility Principle (SRP):
Each method within the ReviewBidsHelper class seems to adhere to the SRP. For example, get_intelligent_topic_row_review_bids is responsible for generating HTML code for topic rows, while get_topic_bg_color_review_bids determines the background color for a topic. This adherence ensures that each method has a clear and distinct purpose, enhancing maintainability and readability.

* Factory Method Pattern:
Although not explicitly labeled as a factory method, get_intelligent_topic_row_review_bids can be interpreted as following a similar pattern. It dynamically generates HTML code based on different scenarios, akin to a factory producing instances of objects. This promotes flexibility and extensibility in generating HTML representations of topics.

After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
* Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

* Link to Testing video: [https://youtu.be/jDfmPUgDDXA]

* Link to pull request : [https://github.com/expertiza/expertiza/pull/2799 here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:49:47Z

Npatil2: /* Links */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
* Single Responsibility Principle (SRP):
Each method within the QuestionnaireHelper class appears to adhere to the SRP by focusing on a single task or responsibility. For example, the adjust_advice_size method is responsible for adjusting the size of advice based on questionnaire scores, while the questionnaire_factory method is responsible for creating instances of questionnaire types based on the given type parameter. This adherence ensures that each method has a clear and distinct purpose, promoting code maintainability and readability.

* Open/Closed Principle (OCP):
While not explicitly evident in the provided snippets, the design allows for extension without modification, which aligns with the OCP. For instance, new types of questionnaires can be added without altering existing code by simply extending the questionnaire_factory method to accommodate the new types.

* Dependency Injection Principle (DIP):
The methods in the QuestionnaireHelper class accept various objects (e.g., questionnaire, scored_question) as parameters, adhering to the principle of dependency injection. By accepting dependencies from external sources rather than creating them internally, these methods become more flexible and easily testable.

* Factory Method Pattern:
The questionnaire_factory method can be seen as exhibiting characteristics of the factory method pattern. It dynamically creates instances of different questionnaire types based on the given type parameter, promoting flexibility and extensibility.

After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
* Single Responsibility Principle (SRP):
Each method within the ReviewBidsHelper class seems to adhere to the SRP. For example, get_intelligent_topic_row_review_bids is responsible for generating HTML code for topic rows, while get_topic_bg_color_review_bids determines the background color for a topic. This adherence ensures that each method has a clear and distinct purpose, enhancing maintainability and readability.

* Factory Method Pattern:
Although not explicitly labeled as a factory method, get_intelligent_topic_row_review_bids can be interpreted as following a similar pattern. It dynamically generates HTML code based on different scenarios, akin to a factory producing instances of objects. This promotes flexibility and extensibility in generating HTML representations of topics.

After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]
Link to Testing video: [https://youtu.be/jDfmPUgDDXA]
Link to pull request : [https://github.com/expertiza/expertiza/pull/2799 here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:46:55Z

Npatil2: /* Objective 1: Develop code testing scenarios for questionnaire_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
* Single Responsibility Principle (SRP):
Each method within the QuestionnaireHelper class appears to adhere to the SRP by focusing on a single task or responsibility. For example, the adjust_advice_size method is responsible for adjusting the size of advice based on questionnaire scores, while the questionnaire_factory method is responsible for creating instances of questionnaire types based on the given type parameter. This adherence ensures that each method has a clear and distinct purpose, promoting code maintainability and readability.

* Open/Closed Principle (OCP):
While not explicitly evident in the provided snippets, the design allows for extension without modification, which aligns with the OCP. For instance, new types of questionnaires can be added without altering existing code by simply extending the questionnaire_factory method to accommodate the new types.

* Dependency Injection Principle (DIP):
The methods in the QuestionnaireHelper class accept various objects (e.g., questionnaire, scored_question) as parameters, adhering to the principle of dependency injection. By accepting dependencies from external sources rather than creating them internally, these methods become more flexible and easily testable.

* Factory Method Pattern:
The questionnaire_factory method can be seen as exhibiting characteristics of the factory method pattern. It dynamically creates instances of different questionnaire types based on the given type parameter, promoting flexibility and extensibility.

After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
* Single Responsibility Principle (SRP):
Each method within the ReviewBidsHelper class seems to adhere to the SRP. For example, get_intelligent_topic_row_review_bids is responsible for generating HTML code for topic rows, while get_topic_bg_color_review_bids determines the background color for a topic. This adherence ensures that each method has a clear and distinct purpose, enhancing maintainability and readability.

* Factory Method Pattern:
Although not explicitly labeled as a factory method, get_intelligent_topic_row_review_bids can be interpreted as following a similar pattern. It dynamically generates HTML code based on different scenarios, akin to a factory producing instances of objects. This promotes flexibility and extensibility in generating HTML representations of topics.

After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]
Link to Testing video: [https://drive.google.com/file/d/1qkfiUrc_NGDQWj7bTXBXNyIqyJAKQO2S/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:45:54Z

Npatil2: /* Objective 2: Develop code testing scenarios for review_bids_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
* Single Responsibility Principle (SRP):
Each method within the ReviewBidsHelper class seems to adhere to the SRP. For example, get_intelligent_topic_row_review_bids is responsible for generating HTML code for topic rows, while get_topic_bg_color_review_bids determines the background color for a topic. This adherence ensures that each method has a clear and distinct purpose, enhancing maintainability and readability.

* Factory Method Pattern:
Although not explicitly labeled as a factory method, get_intelligent_topic_row_review_bids can be interpreted as following a similar pattern. It dynamically generates HTML code based on different scenarios, akin to a factory producing instances of objects. This promotes flexibility and extensibility in generating HTML representations of topics.

After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]
Link to Testing video: [https://drive.google.com/file/d/1qkfiUrc_NGDQWj7bTXBXNyIqyJAKQO2S/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:43:13Z

Npatil2: /* Links */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]
Link to Testing video: [https://drive.google.com/file/d/1qkfiUrc_NGDQWj7bTXBXNyIqyJAKQO2S/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:34:50Z

Npatil2: /* Objective 1: Develop code testing scenarios for questionnaire_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

=== update_questionnaire_questions ===
We did not implement test cases for this method because it was already covered in last semester by previous team.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:29:09Z

Npatil2: /* Coverage Results */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
* Previous coverage: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Current coverage:

[[File:Questionnaire helper spec coverage.jpg]]

=== Review bids helper ===
* Previous coverage: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb
* Current coverage:

[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:27:24Z

Npatil2: /* Questionnaire_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
Previous coverage:
[[File:Questionnaire helper spec coverage.jpg]]
=== Review bids helper ===
Previous coverage:
[[File:Review bid helper spec coverage.jpg]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:26:07Z

Npatil2: /* Coverage Results */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==
=== Questionnaire_helper ===
[[File:Questionnaire helper spec coverage.jpg|thumb|left|Coverage results for questionnaire_helper]]

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

File:Review bid helper spec coverage.jpg

2024-04-24T03:24:49Z

Npatil2:

File:Questionnaire helper spec coverage.jpg

2024-04-24T03:24:12Z

Npatil2:

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:21:16Z

Npatil2: /* Conclusion */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Coverage Results ==

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:20:49Z

Npatil2: /* Conclusion */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Conclusion ==
This document outlines a thorough plan to enhance testing and code coverage for the questionnaire_helper and review_bids_helper files in Expertiza. With defined objectives, including developing detailed test plans and scenarios, the project aims to address current code coverage gaps. Notably, significant improvements were achieved in the review_bids_helper.rb file, with comprehensive test plans substantially increasing code coverage. Conversely, the questionnaire_helper.rb file saw marginal improvements, primarily due to existing full coverage in the update_questionnaire_questions method. Moving forward, the project will focus on implementing the outlined test plans, ensuring comprehensive testing and reliability for critical functionality across both helper files.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:19:29Z

Npatil2: /* Objective 1: Develop code testing scenarios for questionnaire_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* questionnaire_factory
* update_questionnaire_questions

=== adjust_advice_size ===
==== Method Description ====
The .adjust_advice_size method is being described. This method likely adjusts the size of advice related to questions based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a questionnaire, a scored_question, a non_scored_question, and a question_advice.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Question is a ScoredQuestion: This context tests the behavior when the question is a scored question. It sets up expectations related to the adjustment of advice size based on the questionnaire scores.
<pre>
context 'when question is a ScoredQuestion' do
it 'adjusts advice size based on questionnaire scores' do
allow(QuestionAdvice).to receive(:where).and_return([])
allow(QuestionAdvice).to receive(:new).and_return(double('QuestionAdvice', save: true))
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(true)
described_class.adjust_advice_size(questionnaire, scored_question)
expect(QuestionAdvice).to have_received(:where).exactly(10).times
expect(scored_question.question_advices.size).to eq(10)
end
end

</pre>
* When Question is Not a ScoredQuestion: This context tests the behavior when the question is not a scored question. It verifies that in this case, the advice size is not adjusted.
<pre>
context 'when question is not a ScoredQuestion' do
it 'does not adjust advice size' do
allow(QuestionAdvice).to receive(:where)
allow(QuestionAdvice).to receive(:new)
allow(scored_question).to receive(:is_a?).with(ScoredQuestion).and_return(false)
described_class.adjust_advice_size(questionnaire, non_scored_question)
expect(QuestionAdvice).not_to have_received(:where)
expect(QuestionAdvice).not_to have_received(:new)
end
end

</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use allow and expect statements to verify the behavior of the method under different conditions.

=== questionnaire_factory ===

==== Method Description ====
The .questionnaire_factory method is described. It seems to be a factory method responsible for creating instances of different questionnaire types based on the given type parameter.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* When Given a Valid Type: This context verifies the behavior when a valid questionnaire type is provided. It expects that calling questionnaire_factory with a valid type results in an instance of the specified questionnaire type (ReviewQuestionnaire).

<pre>
context 'when given a valid type' do
it 'returns an instance of the specified questionnaire type' do
questionnaire_type = 'ReviewQuestionnaire'
expect(helper.questionnaire_factory(questionnaire_type)).to be_an_instance_of(ReviewQuestionnaire)
end

</pre>
* When Given an Invalid Type: This context tests how the method handles an invalid questionnaire type. It expects that calling questionnaire_factory with an invalid type sets an error flash message indicating that the questionnaire type is undefined.

<pre>
context 'when given an invalid type' do
it 'sets an error flash message' do
questionnaire_type = 'UnknownQuestionnaire'
expect { helper.questionnaire_factory(questionnaire_type) }.to change { flash[:error] }.from(nil).to('Error: Undefined Questionnaire')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect and change statements to verify the behavior of the method under different conditions.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Conclusion ==
The design document outlines a comprehensive plan for the upcoming project focused on enhancing the testing and code coverage for the `questionnaire_helper` and `review_bids_helper` files in Expertiza. With a clear problem statement and objectives defined, the project aims to address the current code coverage gaps by developing thorough test plans and scenarios for both helpers. The document provides an overview of the classes and methods involved. Key objectives include developing code testing scenarios, improving code coverage, and ensuring the reliability and effectiveness of the helpers. Moving forward, the project will involve the implementation of the outlined test plans, execution of test scenarios, and iterative refinement of the codebase to achieve the desired objectives.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-24T03:11:25Z

Npatil2: /* Objective 2: Develop code testing scenarios for review_bids_helper */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===
Our project involves writing test cases for the `questionnaire_helper` and `review_bids_helper` files in Expertiza, an open-source assignment/project management portal built on the Ruby on Rails framework. This platform facilitates collaborative learning and feedback among both instructors and students. Instructors can create and customize assignments, define topics for student sign-ups, and manage teams for various projects. Students, on the other hand, can sign up for topics, form teams, and participate in peer reviews to enhance each other's learning experiences. Our goal is to develop comprehensive test plans and increase code coverage for these helper files to ensure their reliability and effectiveness in the Expertiza platform.

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Objective 1: Develop code testing scenarios for questionnaire_helper ==
We follow the single responsibility principle so that each test only tests
After reviewing the QuestionnaireHelper class, we identified three methods in the class that require testing. These methods are as follows:

* adjust_advice_size
* update_questionnaire_questions
* questionnaire_factory

For testing purposes, we mock the following items:
<pre>
let(:questionnaire) { double('Questionnaire', max_question_score: 5, min_question_score: 1) }
let(:question) { double('ScoredQuestion', id: 1) }
</pre>

=== adjust_advice_size ===
To test this method, we mock a questionnaire and a scored question. We verify that the method correctly adjusts the size of advice for a scored question. We identify the following test cases:

* When the question is a ScoredQuestion: Verify that the method adjusts advice size by ensuring the correct number of advices are created, advices outside the range are deleted, and duplicate advices are removed. Test scenarios when the question's score is within the defined range, above the range, and below the range.
* When the question is not a ScoredQuestion: Verify that the method does not adjust advice size for non-scored questions.

=== update_questionnaire_questions ===
This method updates attributes of questionnaire questions based on form data. To test this method, we mock a question and its associated form data. We identify the following test cases:

* When params contain questions: Verify that the method updates question attributes when parameters contain questions. Test scenarios for updating various attributes of a question.
* When params do not contain questions: Verify that the method does not update any question attributes when parameters do not contain questions.

=== questionnaire_factory ===
This method creates a questionnaire instance based on its type. To test this method, we provide various valid and invalid types. We identify the following test cases:

* When a valid type is provided: Verify that the method creates a questionnaire instance for a valid type. Test scenarios for each valid questionnaire type.
* When an undefined type is provided: Verify that the method sets a flash error message when an undefined questionnaire type is provided.

We have structured the test plan to cover each method comprehensively and ensure that they function correctly under various scenarios.

== Objective 2: Develop code testing scenarios for review_bids_helper ==
After reviewing the ReviewBidsHelper module, we identified two methods that require testing:

=== get_intelligent_topic_row_review_bids ===
==== Objective ====
Verify that the method get_intelligent_topic_row_review_bids correctly renders the topic row for the topics table in review_bids/show.html.erb. The #get_intelligent_topic_row_review_bids method is being described. This method likely generates HTML code for displaying topic rows based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a selected_topic, the number of participants (num_participants), and a review_bid.
Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system. For example, selected_topic is set with a topic ID of 1 and is not waitlisted initially.

==== Test Contexts ====

* When Selected Topics are Present: This context tests the behavior of the method when there are selected topics. It sets up a selected topic that is not waitlisted and expects the generated HTML to include a table row with a yellow background.
<pre>
context 'when selected topics are present' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(false)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="yellow">')
end
end
</pre>
* When Selected Topic is Waitlisted: This context tests the behavior when the selected topic is waitlisted. It sets up a selected topic that is waitlisted and expects the generated HTML to include a table row with a light gray background.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>
* When Selected Topics are Not Present: This context checks what happens when there are no selected topics. It mocks a method (get_topic_bg_color_review_bids) to return a specific background color and expects the generated HTML to include a table row with that background color.
<pre>
context 'when selected topics are not present' do
it 'returns HTML code for topic row with appropriate background color' do
allow(helper).to receive(:get_topic_bg_color_review_bids).and_return('rgb(255,255,255)')
selected_topics = []

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(255,255,255)">')
end
end
</pre>
* When Selected Topics are Nil: This context tests how the method handles cases where the selected topics parameter is nil. It expects the generated HTML to include a table row with a specific background color.
<pre>
context 'when selected topics are nil' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topics = nil

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr id="topic_1" style="background-color:rgb(47,352,0)">')
end
end
end

</pre>
==== Expectations ====
Each context contains an it block with an expectation. These expectations use expect statements to verify that the HTML generated by the method meets certain criteria, such as containing specific table row elements with appropriate background colors.

=== get_topic_bg_color_review_bids ===

==== Objective ====
Verify that the method get_topic_bg_color_review_bids correctly calculates the background color for a topic in the review_bids/show.html.erb template. The #get_topic_bg_color_review_bids method is being described. This method likely determines the background color for a topic based on certain conditions.

==== Test Setup ====
Mock objects are set up using let statements. These include a topic, a review_bid, and the number of participants (num_participants). Each of these mock objects is prepared with predefined attributes that mimic the behavior of actual objects in the system.

==== Test Contexts ====

* Returns RGB Color Code: This test checks if the method returns an RGB color code for the topic background color. It sets up an expectation that the returned color code matches the pattern rgb(\d+,\d+,\d+), indicating it's in the correct format.
<pre>
context 'when there are no review bids' do
it 'returns default RGB color code' do
allow(ReviewBid).to receive(:where).with(signuptopic_id: topic.id).and_return([])

expect(helper.get_topic_bg_color_review_bids(topic, num_participants)).to eq('rgb(47,352,0)')
end
end

</pre>
* When There Are No Review Bids: This context tests the behavior when there are no review bids associated with the topic. It sets up an expectation that the method returns a default RGB color code (rgb(47,352,0)), likely indicating a green color.
<pre>
context 'when selected topic is waitlisted' do
it 'returns HTML code for topic row with appropriate background color' do
selected_topic = instance_double('SelectedTopic')
allow(selected_topic).to receive(:topic_id).and_return(1)
allow(selected_topic).to receive(:is_waitlisted).and_return(true)
selected_topics = [selected_topic]

expect(helper.get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)).to include('<tr bgcolor="lightgray">')
end
end
</pre>

==== Expectations ====
Each test contains an it block with an expectation. These expectations use expect statements to verify the behavior of the method under different conditions.

== Conclusion ==
The design document outlines a comprehensive plan for the upcoming project focused on enhancing the testing and code coverage for the `questionnaire_helper` and `review_bids_helper` files in Expertiza. With a clear problem statement and objectives defined, the project aims to address the current code coverage gaps by developing thorough test plans and scenarios for both helpers. The document provides an overview of the classes and methods involved. Key objectives include developing code testing scenarios, improving code coverage, and ensuring the reliability and effectiveness of the helpers. Moving forward, the project will involve the implementation of the outlined test plans, execution of test scenarios, and iterative refinement of the codebase to achieve the desired objectives.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-09T00:35:42Z

Npatil2: /* Hamer value calculation */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-09T00:34:36Z

Npatil2: /* Objective 2: Develop code testing scenarios for review_bids_helper */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-09T00:33:11Z

Npatil2: /* Objective 2: Develop code testing scenarios for review_bids_helper */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-09T00:26:18Z

Npatil2: /* method 1: adjust_advice_size */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-09T00:05:48Z

Npatil2: /* Objective 1: Develop code testing scenarios for questionnaire_helper */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T17:04:51Z

Npatil2: /* Links */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T17:04:27Z

Npatil2: /* Conclusion */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:59:25Z

Npatil2: /* Conclusion */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:57:32Z

Npatil2: /* Objective 3: Validate the accuracy of the newly implemented Hamer algorithm */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:55:30Z

Npatil2: /* Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:54:49Z

Npatil2: /* Objective 1: Develop code testing scenarios */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:48:34Z

Npatil2: /* Problem Statement */

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:34:27Z

Npatil2: /* Project Overview */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===

=== Current Code Coverage ===
* Questionnaire_helper: 66%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Fquestionnaire_helper.rb
* Review_bids_helper: 17.65%: https://coveralls.io/builds/66490075/source?filename=app%2Fhelpers%2Freview_bids_helper.rb

=== Objectives ===

* Develop test plans/scenarios for questionnaire_helper.rb
* Develop test plans/scenarios for review_bids_helper.rb
* Improve code coverage for questionnaire_helper.rb
* Improve code coverage for review_bids_helper.rb

=== Files Involved ===

* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb
* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As established before, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 3: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
The observed results indicate a tendency towards lower values, primarily due to our decision to include nil values and treat them as zeros in our analysis. This treatment has led to a skew in the scores, favoring lower values and potentially impacting the accuracy of our findings. To address this issue and improve the robustness of our analysis, it is advisable to explore alternative approaches such as using median or random values instead of treating nil values as zeros. However, we must also carefully consider how to handle incomplete reviews that contain nil values in our input dataset, as this can significantly influence the overall integrity and reliability of our results and conclusions.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

It was established that the original reputation web server was implemented incorrectly.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:30:25Z

Npatil2: /* Hamer Algorithm */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===

=== Objectives ===

* bbnn

=== Files Involved ===

* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb
* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Class and Method Overview ==

=== QuestionnaireHelper ===

The QuestionnaireHelper module contains several methods to assist with managing questionnaires in expertiza. QuestionnaireHelper provides methods for adjusting advice size, updating questionnaire questions, and creating questionnaire instances based on types. It also defines constants to facilitate these functionalities. These methods are likely used within the application to handle questionnaire-related tasks efficiently. Let's break down the class and its methods:

==== Constants ====

CSV_QUESTION, CSV_TYPE, CSV_PARAM, CSV_WEIGHT
- These constants define indices for specific columns in a CSV file.

QUESTIONNAIRE_MAP
- This constant is a hash that maps questionnaire types to their respective questionnaire classes.
- It's used by the `questionnaire_factory` method to determine the appropriate class to instantiate.

==== Methods ====

1. adjust_advice_size(questionnaire, question)
- This method adjusts the size of advice associated with a given question in a questionnaire.
- Parameters:
- `questionnaire`: The questionnaire object.
- `question`: The question object whose advice size needs adjustment.
- Functionality:
- Checks if the question is a `ScoredQuestion`.
- Deletes any existing advice for the question outside the score range.
- Iterates over the score range and ensures each score has an associated advice.
- Deletes any duplicate advice records.

2. update_questionnaire_questions
- This method updates attributes of questionnaire questions based on form data, without modifying unchanged attributes.
- Functionality:
- Checks for presence of `params[:question]`.
- Iterates through each question and its attributes in the parameters.
- Compares each attribute's current value with the new value from the parameters and updates if changed.
- Saves the question.

3. questionnaire_factory(type)
- This method acts as a factory to create an appropriate questionnaire object based on the type provided.
- Parameters:
- `type`: The type of questionnaire.
- Functionality:
- Retrieves the questionnaire class from `QUESTIONNAIRE_MAP` based on the provided type.
- If the type is not found in the map, it sets an error flash message.
- Otherwise, it initializes a new instance of the corresponding questionnaire class and returns it.

=== ReviewBidsHelper ===

This Ruby module, `ReviewBidsHelper` serves as a helper module for views related to reviewing bids in expertiza. `ReviewBidsHelper` provides helper methods for rendering topic rows and determining the background color for topics based on their bid status and the number of participants. These methods are likely used in the views associated with reviewing bids in the application. Let's break down the class and its methods:

==== Methods ====

1. get_intelligent_topic_row_review_bids(topic, selected_topics, num_participants)
- This method seems to be responsible for generating HTML markup for a row in a table displaying topics for review bids.
- Parameters:
- `topic`: Represents a specific topic being reviewed.
- `selected_topics`: An array of topics that have been selected.
- `num_participants`: The number of participants involved in the review process.
- Functionality:
- Iterates through the `selected_topics`.
- Depending on the conditions (whether the topic is selected and not waitlisted, or selected and waitlisted), it generates a specific row HTML.
- Returns the generated row HTML as safe HTML.

2. get_topic_bg_color_review_bids(topic, num_participants)
- This method calculates the background color for a topic based on the number of participants and the number of bids for that topic.
- Parameters:
- `topic`: Represents the topic for which the background color is being determined.
- `num_participants`: The total number of participants.
- Functionality:
- Calculates the number of bids for the given `topic`.
- Determines the proportion of bids compared to the total number of participants and adjusts the color accordingly.
- Returns a string representing the RGB value of the background color.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As established before, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 3: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
The observed results indicate a tendency towards lower values, primarily due to our decision to include nil values and treat them as zeros in our analysis. This treatment has led to a skew in the scores, favoring lower values and potentially impacting the accuracy of our findings. To address this issue and improve the robustness of our analysis, it is advisable to explore alternative approaches such as using median or random values instead of treating nil values as zeros. However, we must also carefully consider how to handle incomplete reviews that contain nil values in our input dataset, as this can significantly influence the overall integrity and reliability of our results and conclusions.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

It was established that the original reputation web server was implemented incorrectly.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:15:38Z

Npatil2: /* Files Involved */

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===

=== Objectives ===

* bbnn

=== Files Involved ===

* Review_bids_helper.rb: /app/helpers/review_bids_helper.rb
* Questionnaire_helper.rb: /app/helpers/questionnaire_helper.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As established before, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 3: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
The observed results indicate a tendency towards lower values, primarily due to our decision to include nil values and treat them as zeros in our analysis. This treatment has led to a skew in the scores, favoring lower values and potentially impacting the accuracy of our findings. To address this issue and improve the robustness of our analysis, it is advisable to explore alternative approaches such as using median or random values instead of treating nil values as zeros. However, we must also carefully consider how to handle incomplete reviews that contain nil values in our input dataset, as this can significantly influence the overall integrity and reliability of our results and conclusions.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

It was established that the original reputation web server was implemented incorrectly.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire helper, review bids helper

2024-04-08T16:05:25Z

Npatil2: Created page with "This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper == Project Overview == === Problem Statement === === Objectives === * bbnn === Files Involved === * reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb * test file: /spec/controllers/reputation_mock_web_server_hamer.rb === Mentor === * Muhammet Mustafa Olmez (molmez@ncsu.edu) === Team Members === * Neha Vijay Patil (n..."

This page describes the changes made for the Spring 2024 E2440. Testing for questionnaire_helper, review_bids_helper

== Project Overview ==

=== Problem Statement ===

=== Objectives ===

* bbnn

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As established before, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 3: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
The observed results indicate a tendency towards lower values, primarily due to our decision to include nil values and treat them as zeros in our analysis. This treatment has led to a skew in the scores, favoring lower values and potentially impacting the accuracy of our findings. To address this issue and improve the robustness of our analysis, it is advisable to explore alternative approaches such as using median or random values instead of treating nil values as zeros. However, we must also carefully consider how to handle incomplete reviews that contain nil values in our input dataset, as this can significantly influence the overall integrity and reliability of our results and conclusions.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

It was established that the original reputation web server was implemented incorrectly.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024

2024-04-08T16:03:51Z

Npatil2:

* [[CSC/ECE 517 Spring 2024 - E2401 Implementing and testing import & export controllers]]
* [[CSC/ECE 517 Spring 2024 - E2405 Refactor review_mapping_helper.rb]]
* [[CSC/ECE 517 Spring 2024 - E2407 Refactor review_mapping_controller.rb]]
* [[CSC/ECE 517 Spring 2024 - E2428 Replicate Roles and Institution UIs ReactJS]]
* [[CSC/ECE 517 Spring 2024 - E2429 Reimplement student_task list]]
* [[CSC/ECE 517 Spring 2024 - E2430 Reimplement student_task view]]
* [[CSC/ECE 517 Spring 2024 - E2410. View for Results of Bidding ]]
* [[CSC/ECE 517 Spring 2024 - E2414 Grading Audit Trail]]
* [[CSC/ECE 517 Spring 2024 - NTNX-1 : Extend NDB Operator to Support Postgres HA]]
* [[CSC/ECE 517 Spring 2024 - ‬NTNX-2‬‭ : Snapshot Functionality for provisioned databases]]
* [[CSC/ECE 517 Spring 2024 - E2411 : Fix "Back" link on “New Late Policy” page]]
* [[CSC/ECE 517 Spring 2024 - E2416. Reimplement the Question hierarchy]]
* [[CSC/ECE 517 Spring 2024 - E2420. Reimplement student_quizzes_controller]]
* [[CSC/ECE 517 Spring 2024 - E2424. Reimplement the Bookmarks Controller]]
* [[CSC/ECE 517 Spring 2024 - E2426. Create a UI for Assignment Edit page "Etc" tab in ReactJS]]
* [[CSC/ECE 517 Spring 2024 - E2417. Reimplement submitted content controller.rb]]
* [[CSC/ECE 517 Spring 2024 - E2425. Create a Courses user interface in ReactJS]]
* [[CSC/ECE 517 Spring 2024 - E2431. Reimplement grades/view_team]]
* [[CSC/ECE 517 Spring 2024 - E2404 Refactor student teams functionality]]
* [[CSC/ECE 517 Spring 2024 - E2406 Refactor review_mapping_helper.rb]]
* [[CSC/ECE 517 Spring 2024 - E2403 Mentor-Meeting Management]]
* [[CSC/ECE 517 Spring 2024 - E2421. Reimplement impersonating users (within impersonate controller.rb)]]
* [[CSC/ECE 517 Spring 2024 - E2413. Testing - Answer Tagging]]
* [[CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb]]
* [[CSC/ECE 517 Spring 2024 - E2427. UI for questionnaire.rb]]
* [[CSC/ECE 517 Spring 2024 - E2419. Reimplement duties controller.rb and badges controller.rb]]
* [[CSC/ECE 517 Spring 2024 - E2432. UI for Participants.rb]]
* [[CSC/ECE 517 Spring 2024 - G2402 Implement REST client, REST API, and Graphql API endpoint for repositories]]
* [[CSC/ECE 517 Spring 2024 - G2400 DevOp for GitHub Miner app]]
* [[CSC/ECE 517 Spring 2024 - E2439 Testing for view_translation_substitutor.rb]]
* [[CSC/ECE 517 Spring 2024 - E2443 Reimplement grades_controller]]
* [[CSC/ECE 517 Spring 2024 - E2434 Reimplement Frontend for the Grades view]]
* [[CSC/ECE 517 Spring 2024 - E2435 Implement Frontend for the My Profile]]
* [[CSC/ECE 517 Spring 2024 - E2446 Implement Front End for Student Task List]]
* [[CSC/ECE 517 Spring 2024 - E2444 Implement Frontend for the Review]]
* [[CSC/ECE 517 Spring 2024 - E2440 Testing for questionnaire_helper, review_bids_helper]]

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:34:04Z

Npatil2: /* GitHub Links */

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:32:46Z

Npatil2: /* Conclusion */

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:29:51Z

Npatil2: /* Conclusion */

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:28:05Z

Npatil2: /* Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. */

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:27:39Z

Npatil2: /* Code Snippet */

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:26:12Z

Npatil2: /* Objective 4: Validate the accuracy of the newly implemented Hamer algorithm */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 3: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:23:55Z

Npatil2: /* Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:23:42Z

Npatil2: /* Objective 2: Verify the correctness of the reputation web server's Hamer values */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

File:Results hamer peerlogic.jpg

2024-03-25T04:21:34Z

Npatil2:

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:15:27Z

Npatil2: /* GitHub Links */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1 here]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing here]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:15:15Z

Npatil2: /* GitHub Links */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [https://drive.google.com/file/d/1gZ5iDgqMW3COOT9Uw-_yhJlwJLZDxz-s/view?usp=sharing]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:14:57Z

Npatil2: /* GitHub Links */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [https://github.com/expertiza/expertiza/pull/2778 here]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:10:24Z

Npatil2: /* Conclusion ????? */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ==
In this project, we aimed to test the accuracy of the Hamer algorithm used for assessing the credibility of reviewers in a peer assessment system. We began by developing code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values. These scenarios covered various review scenarios, including cases where reviewers provided extreme scores.

Upon testing the original reputation web server's Hamer values, we found discrepancies between the expected values and the values returned by the web service. This led us to conclude that the webservice implementation was incorrect.

As a result, we proceeded to reimplement the Hamer algorithm in Ruby, incorporating adjustments to handle nil values appropriately. Subsequently, we validated the accuracy of the newly implemented algorithm using the same testing scenarios. While the results initially showed a skew towards lower values due to our treatment of nil values, we acknowledge the need for further refinement to handle these cases more effectively.

In conclusion, this project highlights the importance of rigorous testing and implementation adjustments in ensuring the reliability of algorithms used in peer assessment systems. Moving forward, we recommend further refinements and validations to enhance the accuracy and robustness of the Hamer algorithm.

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [???/]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:08:18Z

Npatil2: /* Conclusion????? */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion ===
As we can see the results are skewed towards lower values. This is because we added the inclusion of nil values and assumed them to be zero. Because of this assumption, the scores are skewed towards lower values. It must be considered median/random to have lesser skew.

== Conclusion ????? ==

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [???/]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T04:06:24Z

Npatil2: /* Results????? */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results===

[[File:Results hamer new.jpeg]]

=== Conclusion????? ===

== Conclusion ????? ==

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [???/]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

File:Results hamer new.jpeg

2024-03-25T04:05:34Z

Npatil2:

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T03:58:01Z

Npatil2: /* Changes made in implementation */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* coded this algorithm in Ruby in a controller.
* Included a way for the algorithm to handle nil values.

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion????? ===

== Conclusion ????? ==

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [???/]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)

CSC/ECE 517 Spring 2024 - E2412. Testing for hamer.rb

2024-03-25T03:53:38Z

Npatil2: /* Objective 2: Verify the correctness of the reputation web server's Hamer values */

This page describes the changes made for the Spring 2024 Program 3: First OSS project E2412. Testing for hamer.rb

== Project Overview ==

=== Problem Statement ===

The practice of using student feedback on assignments as a grading tool is gaining traction among university professors and courses. This approach not only saves instructors and teaching assistants considerable time but also fosters a deeper understanding of assignments among students as they evaluate their peers' work. However, there is a concern that some students may not take their reviewing responsibilities seriously, potentially skewing the grading process by assigning extreme scores such as 100 or 0 arbitrarily. To address this issue, the Hamer algorithm was developed to assess the credibility and accuracy of reviewers. It generates reputation weights for each reviewer, which instructors can use to gauge their reliability or incorporate into grading calculations. Our goal here is to test this Hamer algorithm.

=== Objectives ===

* Develop code testing scenarios to validate the Hamer algorithm and ensure the accuracy of its output values.
* Verify the correctness of the reputation web server's Hamer values by accessing the URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
* Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values.
* Validate the accuracy of the newly implemented Hamer algorithm.

=== Files Involved ===

* reimplemented algorithm: /app/controllers/reputation_web_service_controller.rb
* test file: /spec/controllers/reputation_mock_web_server_hamer.rb

=== Mentor ===

* Muhammet Mustafa Olmez (molmez@ncsu.edu)

=== Team Members ===

* Neha Vijay Patil (npatil2@ncsu.edu)
* Prachit Mhalgi (psmhalgi@ncsu.edu)
* Sahil Santosh Sawant (ssawant2@ncsu.edu)

== Hamer Algorithm ==

The grading algorithm described in the paper is designed to provide a reward to reviewers who participate effectively by allocating a portion of the assignment mark to the review, with the review mark reflecting the quality of the grading. Here's an explanation of the algorithm:

1. Review Allocation: Each reviewer is assigned a number of essays to grade. The paper suggests assigning at least five essays, with ten being ideal. Assuming each review takes 20 minutes, ten reviews can be completed in about three and a half hours.

2. Grading Process:
* Once the reviewing is complete, grades are generated for each essay and weights are assigned to each reviewer.
* The essay grades are computed by averaging the individual grades from all the reviewers assigned to that essay.
* Initially, all reviewers are given equal weight in the averaging process.
* The algorithm assumes that some reviewers will perform better than others. It measures this by comparing the grades assigned by each reviewer to the averaged grades. The larger the difference between the assigned and averaged grades, the more out of step the reviewer is considered with the consensus view of the class.
* The algorithm adjusts the weighting of the reviewers based on this difference. Reviewers who are closer to the consensus view are given higher weights, while those who deviate significantly are given lower weights.

3. Iterative Process:
* The calculation of grades and weights is an iterative process. Each time the grades are calculated, the weights need to be updated, and each change in the weights affects the grades.
* Convergence occurs quickly, typically requiring four to six iterations before a solution (a "fix-point") is reached.

4. Weight Adjustment:
* The weights assigned to reviewers are adjusted based on the difference between the assigned and averaged grades. Reviewers with larger discrepancies have their weights adjusted inversely proportional to this difference.
* To prevent excessively large weights, a logarithmic dampening function is applied, allowing weights to rise to twice the class average before further increases are awarded sparingly.

5. Properties:
* The algorithm aims to identify and diminish the impact of "rogue" reviewers who may inject random or arbitrary grades into the peer assessment process.
* By adjusting reviewer weights based on their grading accuracy, the algorithm aims to improve the reliability of the grading process in the presence of such rogue reviewers.

Overall, the algorithm seeks to balance the contributions of different reviewers based on the accuracy of their grading, ultimately aiming to produce reliable grades for each essay in a peer assessment scenario.

== Hamer value calculation ==
[[File:Step1.PNG|400px]]
 
[[File:Step2.PNG|400px]]
 
[[File:Step3.PNG|400px]]
 
[[File:Step4.PNG|400px]]

== Objective 1: Develop code testing scenarios ==
We assumed 9 reviewers to review 4 submissions each to cover the following test scenarios:
* 3 cases where reviewers are giving credible scores (passing1, passing2, passing3)
* case where reviewer is giving max scores (10) to all submissions (should be flagged)
* case where reviewer is giving min scores (0) to all submissions (should be flagged)
* case where reviewer is giving median scores (5) to all submissions (should be flagged)
* case where reviewer is giving same scores to all submissions (should be flagged)

=== Object Creation ===
Below is the Input object for tests that cover all the above scenarios:

<pre>
INPUTS_new = {
"submission1": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 4,
"max_incomplete": 10,
"sametoall":3,
"passing1": 10,
"passing2": 10,
"passing3": 9
},
"submission2": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"incomplete_review": 2,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 3,
"passing2": 2,
"passing3": 4
},
"submission3": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"sametoall":3,
"passing1": 7,
"passing2": 4,
"passing3": 5
},
"submission4": {
"maxtoall": 10,
"mintoall": 1,
"mediantoall": 5,
"max_incomplete": 10,
"min_incomplete": 1,
"sametoall":3,
"passing1": 6,
"passing2": 4,
"passing3": 5
}
}.to_json
</pre>

=== Expected Hamer Values ===

<pre>
EXPECTED = {
"Hamer": {
"maxtoall": 2.65,
"mintoall": 2.41,
"mediantoall": 1.03,
"incomplete_review": 2.31,
"max_incomplete": 2.57,
"min_incomplete": 2.48,
"sametoall":1.58,
"passing1": 2.17,
"passing2": 1.73,
"passing3": 1.23,
}
}.to_json
</pre>

== Objective 2: Verify the correctness of the reputation web server's Hamer values ==

We test the original reputation web server's algorithm with our scenarios and verify if they match the expected values. The peerlogic server can be accessed via API calls to URL http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms.
It uses two algorithms: The hamer-peer algorithm and the Lauw-peer algorithm. Our scope for this project is to test Hamer values as it is already established in previous work that the Hamer algorithm suits our use case better.

=== Test Code Snippet ===

<pre>
describe "Expertiza" do
it "should return the correct Hamer calculation" do
uri = URI('http://peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms')

response = Net::HTTP.post(uri, INPUTS, 'Content-Type' => 'application/json')

expect(JSON.parse(response.body)["Hamer"]).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion ===

We can see here that the values returned by this webservice don't match the expected values. Hence, we conclude that the webservice is not implemented correctly.

== Objective 3: Reimplement the algorithm if discrepancies arise in the reputation web server's Hamer values. ==

As seen above, the values returned by reputation server do not match the expected values. Hence, we concluded that the PeerLogic Webservice is implemented incorrectly. In this phase, we implemented the algorithm in Ruby as a function in a controller file : /app/controllers/reputation_web_service_controller.rb

=== Changes made in implementation ===
* converted to ruby
* handled nil cases, now it accepts

=== Code Snippet ===
<pre>
require 'json'
require 'uri'
require 'net/http'
require 'openssl'
require 'base64'

# Expertiza allows student work to be peer-reviewed, since peers can provide
# more feedback than the instructor can.
# However, if we want to assure that all students receive competent feedback,
# or even use peer-assigned grades,
# we need a way to judge which peer reviewers are most credible. The solution
# is the reputation system.
# Reputation systems have been deployed as web services, peer-review
# researchers will be able to use them to calculate scores on assignments,
# both past and present (past data can be used to tune the algorithms).
#
# This file is the controller to calculate the reputation scores.
# A 'reputation' measures how close a reviewer's scores are to other reviewers'
# scores.
# This controller implements the calculation of reputation scores.
class ReputationWebServiceController < ApplicationController
include AuthorizationHelper

# Method: action_allowed
# This method checks if the currently authenticated user has the authorization
# to perform certain actions
# Params
#
# Returns
# true if the user has privileges to perform the action else returns false
def action_allowed?
current_user_has_ta_privileges?
end

# Method: get_max_question_score
# This method receives a set of answers and gets the maximum question score
# Params
# answers: set of answers
# Returns
# if no error returns max_question_score of first question else 1
def get_max_question_score(answers)
begin
answers.first.question.questionnaire.max_question_score
rescue StandardError
1
end
end

# Method: get_valid_answers_for_response
# This method receives response and filters the valid answers list of the
# response ID
# Params
# response
# Returns
# set of valid answers (returns nil if empty)
def get_valid_answers_for_response(response)
answers = Answer.where(response_id: response.id)
valid_answer = answers.select { |answer| (answer.question.type == 'Criterion') && !answer.answer.nil? }
valid_answer.empty? ? nil : valid_answer
end

# Method: calculate_peer_review_grade
# This method calculates a cumulative review grade with respect to the set of valid answers
# Params
# valid_answer: valid answer to get weight of the answer's question
# max_question_score: used to calculate maximum score for peer review grade
# Returns
# peer_review_grade
def calculate_peer_review_grade(valid_answer, max_question_score)
weighted_score_sum = valid_answer.map { |answer| answer.answer * answer.question.weight }.inject(:+)
question_weight_sum = valid_answer.sum { |answer| answer.question.weight }
peer_review_grade = 100.0 * weighted_score_sum / (question_weight_sum * max_question_score)
peer_review_grade.round(4)
end

# Method: get_peer_reviews_for_responses
# This method calculates the peer review grade for each valid response
# Params
# reviewer_id: used to create respective element in the peer_review_grades_list
# team_id: used to create respective element in the peer_review_grades_list
# valid_response: to get the valid answer for each valid response
# Returns
# peer_review_grades_list
def get_peer_reviews_for_responses(reviewer_id, team_id, valid_response)
peer_review_grades_list = []
valid_response.each do |response|
valid_answer = get_valid_answers_for_response(response)
next if valid_answer.nil?

review_grade = calculate_peer_review_grade(valid_answer, get_max_question_score(valid_answer))
peer_review_grades_list << [reviewer_id, team_id, review_grade]
end
peer_review_grades_list
end

# Method: get_peer_reviews
# This method retrieves all the reviews for the submissions
# Params
# assignment_id_list: used to retrieve response map
# round_num: used to retrieve round_num for the valid response
# has_topic: to get the topic condition
# Returns
# raw_data_array: which corresponds to the return of
# get_peer_reviews_for_responses method and appended to the raw_data_array
def get_peer_reviews(assignment_id_list, round_num, has_topic)
raw_data_array = []
ReviewResponseMap.where('reviewed_object_id in (?) and calibrate_to = ?', assignment_id_list, false).each do |response_map|
reviewer = response_map.reviewer.user
team = AssignmentTeam.find(response_map.reviewee_id)
topic_condition = ((has_topic && (SignedUpTeam.where(team_id: team.id).first.is_waitlisted == false)) || !has_topic)
last_valid_response = response_map.response.select { |r| r.round == round_num }.max
valid_response = [last_valid_response] unless last_valid_response.nil?
if (topic_condition == true) && !valid_response.nil? && !valid_response.empty?
raw_data_array += get_peer_reviews_for_responses(reviewer.id, team.id, valid_response)
end
end
raw_data_array
end

# Method: get_ids_list
# This method maps each object to the corresponding object's ID
# Params
# tables: any table
# Returns
# id in the tables
def get_ids_list(tables)
tables.map(&:id)
end

# Method: get_scores
# This method gets the quiz score of each participant for respective reviewee
# Params
# team_ids: list of team IDs
# Returns
# raw_data_array: which is a list of participant, reviewee and the participant's quiz score
def get_scores(team_ids)
quiz_questionnnaires = QuizQuestionnaire.where('instructor_id in (?)', team_ids)
quiz_questionnnaire_ids = get_ids_list(quiz_questionnnaires)
raw_data_array = []
QuizResponseMap.where('reviewed_object_id in (?)', quiz_questionnnaire_ids).each do |response_map|
quiz_score = response_map.quiz_score
participant = Participant.find(response_map.reviewer_id)
raw_data_array << [participant.user_id, response_map.reviewee_id, quiz_score]
end
raw_data_array
end

# Method: get_quiz_score
# This method gets the quiz score of assignments
# Params
# assignment_id_list: list of assignment IDs
# Returns
# raw_data_array: returned by get_scores method, which is a list of participant,
# reviewee and the participant's quiz score
def get_quiz_score(assignment_id_list)
teams = AssignmentTeam.where('parent_id in (?)', assignment_id_list)
team_ids = get_ids_list(teams)
get_scores(team_ids)
end

# Method: generate_json_body
# This method generates json body for the peer reviews and quiz scores
# Params
# results: list of grades with corresponding team/participant ID,
# reviewee ID and their score
# Returns
# request_body: returns the formatted body after sorting the hash
def generate_json_body(results)
request_body = {}
results.each_with_index do |record, _index|
request_body['submission' + record[1].to_s] = {} unless request_body.key?('submission' + record[1].to_s)
request_body['submission' + record[1].to_s]['stu' + record[0].to_s] = record[2]
end
# sort the 2-dimension hash
request_body.each { |k, v| request_body[k] = v.sort.to_h }
request_body.sort.to_h
request_body
end

# Method: generate_json_for_peer_reviews
# This method retrieves all the peer reviews associated with
# the assignment id list by calling the get_peer_reviews method.
# It then formats the peer-review list in JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# round_num: round number of the review
# Returns
# request_body: request body populated with the formatted peer review data.
def generate_json_for_peer_reviews(assignment_id_list, round_num = 2)
has_topic = !SignUpTopic.where(assignment_id: assignment_id_list[0]).empty?

peer_reviews_list = get_peer_reviews(assignment_id_list, round_num, has_topic)
request_body = generate_json_body(peer_reviews_list)
request_body
end

# Method: generate_json_for_quiz_scores
# This method accepts a list of assignment ids as an argument.
# It then calls the get_quiz_score method on the list to get
# maps of teams and scores for the given assignments.
# The map is then formatted into JSON.
# Params
# assignment_id_list: list of assignment ids to get quiz scores for
# Returns
# request_body: request body populated with quiz scores
def generate_json_for_quiz_scores(assignment_id_list)
participant_reviewee_map = get_quiz_score(assignment_id_list)
request_body = generate_json_body(participant_reviewee_map)
request_body
end

# Method: client
# This method is called when the url reputation_web_service/client
# is hit using GET method.
# This renders the client.html.erb
# It also populates the instance variables to be used in the views
# Params
#
# Returns
# nil
def client
@max_assignment_id = Assignment.last.id
@assignment = Assignment.find(flash[:assignment_id]) rescue nil
@another_assignment = Assignment.find(flash[:another_assignment_id]) rescue nil
end

# Method: update_participants_reputation
# This method accepts the response body in the JSON format.
# It then parses the JSON and updates the reputation scores of the
# participants in the list.
# If the alg variable is not Hamer/ Lauv, the updation step is skipped.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def update_participants_reputation(reputation_response)
JSON.parse(reputation_response.body.to_s).each do |reputation_algorithm, user_resputation_list|
next unless %w[Hamer Lauw].include?(reputation_algorithm)

user_resputation_list.each do |user_id, reputation|
Participant.find_by(user_id: user_id).update(reputation_algorithm.to_sym => reputation) unless /leniency/ =~ user_id.to_s
end
end
end

# Method: process_response_body
# This method gets the control after receiving a response from the server.
# It receives the response body as an argument
# It updates the instance variables related to the response.
# It then calls the update_participants_reputation to update the reputation
# scores received in the response body.
# Params
# reputation_response: The response from the reputation web service
# Returns
# nil
def process_response_body(reputation_response)
flash[:response] = reputation_response
flash[:response_body] = reputation_response.body
update_participants_reputation(reputation_response)
end

# Method: add_expert_grades
# It prepends the request body with the expert grades pertaining
# to the default wiki contribution case of 754.
# It receives the request body as an argument and prepends it
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_expert_grades(body)
flash[:additional_info] = 'add expert grades'
case params[:assignment_id]
when '754' # expert grades of Wiki contribution (754)
body.prepend('"expert_grades": {"submission25030":95,"submission25031":92,"submission25033":88,"submission25034":98,"submission25035":100,"submission25037":95,"submission25038":95,"submission25039":93,"submission25040":96,"submission25041":90,"submission25042":100,"submission25046":95,"submission25049":90,"submission25050":88,"submission25053":91,"submission25054":96,"submission25055":94,"submission25059":96,"submission25071":85,"submission25082":100,"submission25086":95,"submission25097":90,"submission25098":85,"submission25102":97,"submission25103":94,"submission25105":98,"submission25114":95,"submission25115":94},')
end
end

# Method: add_quiz_scores
# It gets the assignment id list and generates the json on quiz scores of
# those assignments.
# Finally processes quiz string is prepended to the request body, received
# as an argument, and returns the body to prepare_request_body.
# Params
# body: The request body to add the expert grades to
# Returns
# body prepended with the expert grades
def add_quiz_scores(body)
flash[:additional_info] = 'add quiz scores'
assignment_id_list_quiz = get_assignment_id_list(params[:assignment_id].to_i, params[:another_assignment_id].to_i)
quiz_str = generate_json_for_quiz_scores(assignment_id_list_quiz).to_json
quiz_str[0] = '' # remove first {
quiz_str.prepend('"quiz_scores":{') # add quiz_scores tag
quiz_str += ','
quiz_str = quiz_str.gsub('"N/A"', '20.0') # replace N/A values with 20
body.prepend(quiz_str)
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_hamer_reputation_values
flash[:additional_info] = 'add initial hamer reputation values'
end

# Method: add_lauw_reputation_values
# This method sets the instance variable @additional_info.
# This method is called by the prepare_request_body method
# when params receive instruction through the corresponding view's checkbox.
# THIS METHOD IS NOT IMPLETEMENTED
# Params
#
# Returns
# nil
def add_lauw_reputation_values
flash[:additional_info] = 'add initial lauw reputation values'
end

# Method: get_assignment_id_list
# This method on receipt of individual assignment IDs returns a list with all
# the assignment IDs appended into a data structure
# This function accepts 2 arguments, with the second argument being optional,
# and returns the list assignment_id_list
# If the second argument is 0, it is not appended to the list.
# Params
# assignment_id_one: first assignment id (required)
# assignment_id_two: second assignment id (optional)
# Returns
# assignment_id_list: list containing two assignment ids
def get_assignment_id_list(assignment_id_one, assignment_id_two = 0)
assignment_id_list = []
assignment_id_list << assignment_id_one
assignment_id_list << assignment_id_two unless assignment_id_two.zero?
assignment_id_list
end

# Method: add_flash_messages
# This method sets the flash messages to pass on to the next request i.e
# the request redirected to the client
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_flash_messages(post_req)
flash[:assignment_id] = params[:assignment_id]
flash[:round_num] = params[:round_num]
flash[:algorithm] = params[:algorithm]
flash[:another_assignment_id] = params[:another_assignment_id]
flash[:request_body] = post_req.body
end

# Method: add_additional_info_details
# This method sets the additional info details based on the options
# selected in the additional information section. We populate the request
# based on the selections
# Params
# post_req: This contains the entire post_req that needs to be sent to the reputation
# webservice
# Returns
# nil
def add_additional_info_details(post_req)
if params[:checkbox][:expert_grade] == 'Add expert grades'
add_expert_grades(post_req.body)
elsif params[:checkbox][:hamer] == 'Add initial Hamer reputation values'
add_hamer_reputation_values
elsif params[:checkbox][:lauw] == 'Add initial Lauw reputation values'
add_lauw_reputation_values
elsif params[:checkbox][:quiz] == 'Add quiz scores'
add_quiz_scores(post_req.body)
else
flash[:additional_info] = ''
end
end

# Method: prepare_request_body
# This method is responsible for preparing the request body in a proper format
# to send to the server. It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# It finally sends the prepared request body back to the send_post_request
# method.
# Params
#
# Returns
# nil
def prepare_request_body
reputation_web_service_path = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).path
post_req = Net::HTTP::Post.new(reputation_web_service_path, { 'Content-Type' => 'application/json', 'charset' => 'utf-8' })
curr_assignment_id = (params[:assignment_id].empty? ? '754' : params[:assignment_id])
assignment_id_list_peers = get_assignment_id_list(curr_assignment_id, params[:another_assignment_id].to_i)

post_req.body = generate_json_for_peer_reviews(assignment_id_list_peers, params[:round_num].to_i).to_json

post_req.body[0] = '' # remove the first '{'
add_additional_info_details post_req
post_req.body.prepend('{')
add_flash_messages post_req
post_req
end

# Method: send_post_request
# This method calls the prepare_request_body function to get a prepared
# request body in proper format to send to the server.
# It populates the assignment scores and peer review
# scores. It also populates the flash messages to send to the next request
# We redirect to the client url to display the results.
# Params
#
# Returns
# nil
def send_post_request
post_req = prepare_request_body
reputation_web_service_hostname = URI.parse(WEBSERVICE_CONFIG['reputation_web_service_url']).host
reputation_response = Net::HTTP.new(reputation_web_service_hostname).start { |http| http.request(post_req) }
if %w[400 500].include?(reputation_response.code)
flash[:error] = 'Post Request Failed'
else
process_response_body(reputation_response)
end
redirect_to action: 'client'
end

def calculate_reputation_Score
end

# Method: calculate_reputation_score
# This method calculates the reputation scores for each reviewer based on the provided input data.
# It first parses the input JSON string to extract the submissions and their corresponding scores.
# Then, it calculates the average weighted grades per reviewer and the delta R values.
# Next, it calculates the weight prime values based on the delta R values.
# Finally, it calculates the reputation weights for each reviewer using the weight prime values.
#
# Params
# - input_json: a JSON string representing the input data with submission scores
#
# Returns
# An array of reputation scores, one score per reviewer, indicating their reputation in the system.
def calculate_reputation_score(reviews)
# Parse the input JSON string
# reviews = JSON.parse(input_json)

# Initialize arrays to store intermediate values
grades = []
delta_r = []
weight_prime = []
weight = []

# Calculate Average Weighted Grades per Reviewer
reviews.each do |reviewer_marks|
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
assignment_grade_average = reviewer_marks_without_nil.sum.to_f / reviewer_marks_without_nil.length
grades << assignment_grade_average
end

# Calculate delta R
reviews.each do |reviewer_marks|
reviewer_delta_r = 0
# Skip nil values when calculating the sum
reviewer_marks_without_nil = reviewer_marks.compact
reviewer_marks_without_nil.each_with_index do |grade, student_index|
reviewer_delta_r += (grade - grades[student_index]) ** 2
end
delta_r << reviewer_delta_r / reviewer_marks_without_nil.length
end

# Calculate weight prime
average_delta_r = delta_r.sum / delta_r.length.to_f

delta_r.each do |reviewer_delta_r|
weight_prime << average_delta_r / reviewer_delta_r
end

# Calculate reputation weight
weight_prime.each do |reviewer_weight_prime|
if reviewer_weight_prime <= 2
weight << reviewer_weight_prime.round(2)
else
weight << (2 + Math.log(reviewer_weight_prime - 1)).round(2)
end
end

# Return the reputation weights
weight
end
end
</pre>

== Objective 4: Validate the accuracy of the newly implemented Hamer algorithm ==

We test the newly implemented Hamer algorithm function with our scenarios and verify if they match the expected values.

=== Test Code Snippet ===

<pre>
describe ReputationWebServiceController do
it "should calculate correct Hamer calculation" do
weights = ReputationWebServiceController.new.calculate_reputation_score(reviews)
keys = ["maxtoall", "mintoall", "mediantoall", "incomplete_review", "sametoall", "passing1", "passing2", "passing3"]
rounded_weights = weights.map { |w| w.round(1) }
result_hash = keys.zip(rounded_weights).to_h
expect(result_hash).to eq(JSON.parse(EXPECTED)["Hamer"])
end
end
</pre>

=== Results????? ===

=== Conclusion????? ===

== Conclusion ????? ==

==GitHub Links==
Link to Expertiza repository: [https://github.com/expertiza/expertiza here]

Link to the forked repository: [https://github.com/Prachit99/expertiza/tree/main here]

Link to pull request: [???/]

Link to Github Project page: [https://github.com/users/Prachit99/projects/1]

Link to Testing Video: [????]

== References ==

1. Expertiza on GitHub (https://github.com/expertiza/expertiza) 
2. The live Expertiza website (http://expertiza.ncsu.edu/) 
3. Pluggable reputation systems for peer review: A web-service approach (https://doi.org/10.1109/FIE.2015.7344292)