Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2021 - E2169. Testing - Answer Tagging

2021-12-07T18:23:59Z

Akkhadse: /* Controller: answer_tags_controller.rb */

__TOC__

== Background ==
Answer tagging (see [https://github.com/expertiza/expertiza/blob/25713567e5e5e1c0b363409715e6d8797b45b01a/app/models/answer_tag.rb answer_tab.rb] and [[Answer_tags|wiki]]) helps determine a few metrics on a student’s responses to a questionnaire. Students can tag these metrics for other students for things such as “Positive Tone” or “Suggest Solutions”.

These specs below all need to be further developed to cover missing methods/lines. Some structures currently exist at the top of spec files (such as assignments, participants, etc) to help make writing tests easier. Please feel free to add to these and write extra tests to cover any edge cases you may think of.

As an instructor on an assignment, going to Etc/View Reports/Answer Tagging Report will show student answer tagging. You will be able to see the percentage of answers tagged, # of answers tagged, # not tagged, and # of taggable answers for each student on a questionnaire.

[[File:Tagging_report.png|thumb|center|Fig. 1]]

== Project Purpose==
The objective of the project is unit testing of functionality in these two ruby files and increasing the coverage.

[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec]

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec]

== Initial Coverage ==
[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec] : 0% Covered

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec] : 18% Covered

=== Missed Methods ===

These methods had no tests before our project was done.

* <code>app/controllers/answer_tags_controller.rb</code>
** <code>action_allowed?</code>
** <code>index</code>
** <code>create_edit</code>
** <code>destroy</code>

* <code>app/models/tag_prompt_deployment.rb</code>
** <code>get_number_of_taggable_answers</code>
** <code>assignment_tagging_progress</code>

== Project Design ==
In this project, we have two controllers to be tested using Rspec Unit testing.

The ER diagram of the parent tables is given below for reference ([https://expertiza.csc.ncsu.edu/index.php/Answer_tags Source: Answer_tags]).

[[File:answer_tags_imported.png]]

The flow diagrams below show how controllers in this project are connected with interrelated classes, methods, and views.

[[File:E2169 answer tags controller.png|900px]]

[[File:E2169 tag prompt deployment.png|900px]]

== Testing Plan ==
=== Controller: answer_tags_controller.rb ===

Existing coverage = 0%

Description:
The purpose of this controller is to track down the total number of tags done by the students. The output of the controller is the list of answer tags in JSON format. The database structure for the answer_tags can be found here [https://expertiza.csc.ncsu.edu/index.php/Answer_tags]

'''Methods'''

1. action_allowed? ( params: action )

Description: To allow the functionality only if the accessing user is having student privileges (current_user_has_student_privileges?)

<code>Use cases:</code>
* <code>When the session is of student type, access should be allowed and index operation should be executed.</code>
* <code>When the session is of student type, access should be allowed and create_edit operation should be executed.</code>

<code>Edge cases:</code>
* <code>When the session is null, Access should be denied for index</code>
* <code>When the session is null, Access should be denied for create_edit</code>

2. index ( params : assignment_id, questionnaire_id, user_id )

Description: Populates the tags by fetching the records from AnswerTag corresponding to each TagPromptDeployment based on the filers provided as params.

<code>Use cases:</code>
* <code> When all valid params are passed, the function should populate the answer tags.</code>
* <code> When user_id is provided, answer tags specific to that user should only be populated.</code>
* <code> When questionnaire_id is provided, answer tags specific to that questionnaire should only be populated.</code>
* <code> When assignment_id is provided, answer tags specific to that assignment should only be populated.</code>
* <code> When only one parameter is passed, the corresponding answer tags should be populated.</code>

<code>Edge test cases:</code>
* <code> When user_id is nil, 0 tags prompts should be returned.</code>
* <code> When questionnaire_id is nil, 0 tags prompts should be returned.</code>
* <code> When assignment_id is nil, 0 tags prompts should be returned.</code>

3. create_edit ( params : answer_id, tag_prompt_deployment_id, value )

Description: Fetches the answer tag with the params provided and then updates the respective tag values by the updated values provided as params.

<code>Use cases:</code>
* <code> When the existing tag is found based on the params passed, the existing tag should be updated with new values.</code>

<code>Edge cases:</code>
* <code> Restricts updating answer tag by the student if no mapping is found related to any answer for that tag (foreign key constraint).</code>
* <code> Restricts updating answer tag by the student if no mapping is found related to any tag_prompt_deployment for that tag (foreign key constraint).</code>
* <code> Restricts updating answer tag by student if no updated value is provided for the answer tag.</code>

=== Controller: tag_prompt_deployment.rb ===

Existing coverage = 18%

Description:
This Controller is when a user (with instructor privilege) wants to see tagging summary of every user that participated in a tagging assignment.

'''Methods'''

1. tag_prompt
Description: tag_prompt has existing test cases scoring 18 percentage coverage. Additional testing is not required.

2. get_number_of_taggable_answers ( params: user_id)

Description: To calculate total taggable answers reviewed by a user that participated in the given tagging assignment. user_id is the parameter passed to this method. An individual and his team will have the same score. Fig 1: The last column values are processed by this method.

<code>Use cases:</code>
* <code>when user_id is nil expect an error</code>
* <code>when a team has given any response then count of taggable answer > 0</code>
* <code>When a question or a response does not exists then count of taggable answer = 0 (extreme case) </code>
* <code>When answer_length_threshold not set, count taggable answer</code>
* <code>When answer_length_threshold is set, count taggable answer</code>

<code>Edge cases:</code>
* user_id (parameter) is nil
* response is empty
* question is empty

3. assignment_tagging_progress

Description: To compute the number of tagged answers, number of untagged answers, and percentage of answers tagged by the users that participated in the tagging Assignment to a particular question.
Fig 1: First three columns after the 2nd column onwards are updated with results obtained from this method.

<code>Use cases:</code>
* <code>When not team participated in the assignment, assert count(user_answer_tagging) =0</code>
* <code>When the question does not exist, assert count(user_answer_tagging) =0 </code>
* <code>When multiple rounds of review, render ReviewResponseMap and assert response object</code>
* <code>When single round of review, render ResponseMap and assert response object</code>
* <code>When answer_length_threshold is SET, number of answers and when answer_length_threshold is NOT SET, number of answers</code>
* <code>When route is success answer_tagging object structure matching</code>

<code>Edge cases:</code>
* There are no teams associated with the assignment.
* There are no questions in the questionnaire that have the question type specified in the TagPromptDeployment object.
* The assignment is set to vary by round.
* The assignment is set not to vary by round.
* There are multiple tags.
* There are no tags.

== Bug Fixing of Additional Issues ==

===Model: tag_prompt_deployment.rb (method: get_number_of_taggable_answers)===

An additional issue in the method get_number_of_taggable_answers. The error message is as shown below which indicates string concatenation fails between integer and string data types.

''TypeError:''
no implicit conversion of Fixnum into String at line 23 https://github.com/expertiza/expertiza/blob/beta/app/models/tag_prompt_deployment.rb.

[[File:Get_number_of_taggable_answers_bug.JPG]]

This issue is fixed by correcting the syntax of the two-strings concatenation. After fixing the code is

<code> answers = answers.where(conditions: "length(comments) < #{self.answer_length_threshold}" ) unless self.answer_length_threshold.nil? </code>

===Model: tag_prompt_deployment.rb (method: assignment_tagging_progress)===

<pre>
Failure/Error: answers = answers.where("length(comments) > ?", self.answer_length_threshold.to_s) unless self.answer_length_threshold.nil?

NoMethodError:
undefined method `where' for #<Array:0x00000003424b60>
</pre>

This issue was resolved by using <code>Arrays.select</code> to achieve the same functionality.

There was another small error in which the TeamsUser <code>user</code> was not found, and the resolution was to find the User model for each <code>TeamsUser.user_id</code>.

== Results ==
Test cases are added to cover all functionality. With the cases added the coverage has reached to 100%.

Test cases demo can be found at [https://youtu.be/WYSUmL6jcHc OSS Final Project Testing Video]

tag_prompt_deployment.rb: Coverage = 100%

[[File:Tag_prompt_deployment_coverage.jpeg]]

answer_tags_controller.rb: Coverage = 100%

[[File:Answer_tag_controller_coverage.jpeg]]

== Collaborators ==
Nicholas Himes (Mentor)

1. Aditya Khadse

2. Alec Landow

3. Rageeni Sah

4. Sayali Parab

== Relevant Links ==
Main Expertiza repository can be found at [https://github.com/expertiza/expertiza/tree/beta GitHubExp]

Our forked repository can be found at [https://github.com/akk5597/expertiza Github]

== Video Link ==

Test case run and coverage recording [https://www.youtube.com/watch?v=WYSUmL6jcHc]

CSC/ECE 517 Fall 2021 - E2169. Testing - Answer Tagging

2021-11-30T01:49:10Z

Akkhadse: /* Controller: answer_tags_controller.rb */

__TOC__

== Background ==
Answer tagging (see [https://github.com/expertiza/expertiza/blob/25713567e5e5e1c0b363409715e6d8797b45b01a/app/models/answer_tag.rb answer_tab.rb] and [[Answer_tags|wiki]]) helps determine a few metrics on a student’s responses to a questionnaire. Students can tag these metrics for other students for things such as “Positive Tone” or “Suggest Solutions”.

These specs below all need to be further developed to cover missing methods/lines. Some structures currently exist at the top of spec files (such as assignments, participants, etc) to help make writing tests easier. Please feel free to add to these and write extra tests to cover any edge cases you may think of.

As an instructor on an assignment, going to Etc/View Reports/Answer Tagging Report will show student answer tagging. You will be able to see the percentage of answers tagged, # of answers tagged, # not tagged, and # of taggable answers for each student on a questionnaire.

[[File:Tagging_report.png|thumb|center|Fig. 1]]

== Project Purpose==
The objective of the project is unit testing of functionality in these two ruby files and increasing the coverage.

[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec]

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec]

== Initial Coverage ==
[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec] : 0% Covered

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec] : 18% Covered

=== Missed Methods ===
* <code>app/controllers/answer_tags_controller.rb</code>
** <code>action_allowed?</code>
** <code>index</code>
** <code>create_edit</code>
** <code>destroy</code>

* <code>app/models/tag_prompt_deployment.rb</code>
** <code>get_number_of_taggable_answers</code>
** <code>assignment_tagging_progress</code>

== Project Design ==
In this project, we have two controllers to be tested using Rspec Unit testing.

The ER diagram of the parent tables is given below for reference ([https://expertiza.csc.ncsu.edu/index.php/Answer_tags Source: Answer_tags]).

[[File:answer_tags_imported.png]]

The flow diagrams below show how controllers in this project are connected with interrelated classes, methods, and views.

[[File:E2169 answer tags controller.png|900px]]

[[File:E2169 tag prompt deployment.png|900px]]

== Testing Plan ==
=== Controller: answer_tags_controller.rb ===

Existing coverage = 0%

Description:
The purpose of this controller is to track down the total number of tags done by the students. The database structure for the answer_tags can be found here [https://expertiza.csc.ncsu.edu/index.php/Answer_tags]

'''Methods'''

1. action_allowed? ( params: action )

Description: To allow the functionality only if the accessing user is having student privileges (current_user_has_student_privileges?)

<code>Use cases:</code>
* <code>When the session is null, Access should be denied</code>
* <code>When the session is of student type, access should be allowed and operation should be executed.</code>

2. index ( params : assignment_id, questionnaire_id, user_id )

Description: Populates the tags by fetching the records from AnswerTag corresponding to each TagPromptDeployment based on the filers provided as params.

<code>Use cases:</code>
* <code> When all valid params are passed, the function should populate the answer tags.</code>
* <code> When user_id is provided, answer tags specific to that user should only be populated.</code>
* <code> When questionnaire_id is provided, answer tags specific to that questionnaire should only be populated.</code>
* <code> When assignment_id is provided, answer tags specific to that assignment should only be populated.</code>
* <code> When only one parameter is passed, the corresponding answer tags should be populated.</code>

<code>Edge test cases:</code>
* <code> When user_id is nil, 0 tags prompts should be returned.</code>
* <code> When questionnaire_id is nil, 0 tags prompts should be returned.</code>
* <code> When assignment_id is nil, 0 tags prompts should be returned.</code>

3. create_edit ( params : answer_id, tag_prompt_deployment_id, value )

Description: Fetches the answer tag with the params provided and then updates the respective tag values by the updated values provided as params.

<code>Use cases:</code>
* <code> When the existing tag is found based on the params passed, the existing tag should be updated with new values.</code>

=== Controller: tag_prompt_deployment.rb ===

Existing coverage = 18%

Description:
This Controller is when a user (with instructor privilege) wants to see tagging summary of every user that participated in a tagging assignment.

'''Methods'''

1. tag_prompt
Description: tag_prompt has existing test cases scoring 18 percentage coverage. Additional testing is not required.

2. get_number_of_taggable_answers ( params: user_id)

Description: To calculate total taggable answers reviewed by a user that participated in the given tagging assignment. user_id is the parameter passed to this method. An individual and his team will have the same score. Fig 1: The last column values are processed by this method.

<code>Use cases:</code>
* <code>when a team has given any response then count of taggable answer > 0</code>
* <code>When a question or a response does not exists then count of taggable answer = 0 (extreme case) </code>
* <code>When answer_length_threshold not set, count taggable answer</code>
* <code>When answer_length_threshold is set, count taggable answer</code>

3. assignment_tagging_progress

Description: To compute the number of tagged answers, number of untagged answers, and percentage of answers tagged by the users that participated in the tagging Assignment to a particular question.
Fig 1: First three columns after the 2nd column onwards are updated with results obtained from this method.

<code>Use cases:</code>
* <code>When not team participated in the assignment, assert count(user_answer_tagging) =0</code>
* <code>When the question does not exist, assert count(user_answer_tagging) =0 </code>
* <code>When multiple rounds of review, render ReviewResponseMap and assert response object</code>
* <code>When single round of review, render ResponseMap and assert response object</code>
* <code>When answer_length_threshold is SET, number of answers and when answer_length_threshold is NOT SET, number of answers</code>
* <code>When route is success answer_tagging object structure matching</code>

<code>Edge cases:</code>
* There are no teams associated with the assignment.
* There are no questions in the questionnaire that have the question type specified in the TagPromptDeployment object.
* The assignment is set to vary by round.
* The assignment is set not to vary by round.
* There are multiple tags.

== Bug Fixing of Additional Issues ==

===Model: tag_prompt_deployment.rb (method: get_number_of_taggable_answers)===

An additional issue in the method get_number_of_taggable_answers. The error message is as shown below which indicates string concatenation fails between integer and string data types.

''TypeError:''
no implicit conversion of Fixnum into String at line 23 https://github.com/expertiza/expertiza/blob/beta/app/models/tag_prompt_deployment.rb.

[[File:Get_number_of_taggable_answers_bug.JPG]]

This issue is fixed by correcting the syntax of the two-strings concatenation. After fixing the code is

<code> answers = answers.where(conditions: "length(comments) < #{self.answer_length_threshold}" ) unless self.answer_length_threshold.nil? </code>

===Model: tag_prompt_deployment.rb (method: assignment_tagging_progress)===

<pre>
Failure/Error: answers = answers.where("length(comments) > ?", self.answer_length_threshold.to_s) unless self.answer_length_threshold.nil?

NoMethodError:
undefined method `where' for #<Array:0x00000003424b60>
</pre>

This issue was resolved by using <code>Arrays.select</code> to achieve the same functionality.

There was another small error in which the TeamsUser <code>user</code> was not found, and the resolution was to find the User model for each <code>TeamsUser.user_id</code>.
A similar issue was found with TeamsUser

== Results ==
Test cases are added to cover all functionality. With test cases the new coverage is as follows:

tag_prompt_deployment.rb: Coverage = 99%

answer_tags_controller.rb: Coverage = 100%

Edge cases that are considered in the project are:

-
-
-

== Collaborators ==
Nicholas Himes (Mentor)

1. Aditya Khadse

2. Alec Landow

3. Rageeni Sah

4. Sayali Pranab

== Relevant Links ==
Main Expertiza repository can be found at [https://github.com/expertiza/expertiza/tree/beta GitHubExp]

Our forked repository can be found at [https://github.com/akk5597/expertiza Github]

CSC/ECE 517 Fall 2021 - E2169. Testing - Answer Tagging

2021-11-30T01:48:44Z

Akkhadse: /* Testing Plan */

__TOC__

== Background ==
Answer tagging (see [https://github.com/expertiza/expertiza/blob/25713567e5e5e1c0b363409715e6d8797b45b01a/app/models/answer_tag.rb answer_tab.rb] and [[Answer_tags|wiki]]) helps determine a few metrics on a student’s responses to a questionnaire. Students can tag these metrics for other students for things such as “Positive Tone” or “Suggest Solutions”.

These specs below all need to be further developed to cover missing methods/lines. Some structures currently exist at the top of spec files (such as assignments, participants, etc) to help make writing tests easier. Please feel free to add to these and write extra tests to cover any edge cases you may think of.

As an instructor on an assignment, going to Etc/View Reports/Answer Tagging Report will show student answer tagging. You will be able to see the percentage of answers tagged, # of answers tagged, # not tagged, and # of taggable answers for each student on a questionnaire.

[[File:Tagging_report.png|thumb|center|Fig. 1]]

== Project Purpose==
The objective of the project is unit testing of functionality in these two ruby files and increasing the coverage.

[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec]

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec]

== Initial Coverage ==
[https://coveralls.io/builds/43656335/source?filename=app%2Fcontrollers%2Fanswer_tags_controller.rb Answer_tags_controller_spec] : 0% Covered

[https://coveralls.io/builds/43656335/source?filename=app%2Fmodels%2Ftag_prompt_deployment.rb Tag_prompt_deployment_spec] : 18% Covered

=== Missed Methods ===
* <code>app/controllers/answer_tags_controller.rb</code>
** <code>action_allowed?</code>
** <code>index</code>
** <code>create_edit</code>
** <code>destroy</code>

* <code>app/models/tag_prompt_deployment.rb</code>
** <code>get_number_of_taggable_answers</code>
** <code>assignment_tagging_progress</code>

== Project Design ==
In this project, we have two controllers to be tested using Rspec Unit testing.

The ER diagram of the parent tables is given below for reference ([https://expertiza.csc.ncsu.edu/index.php/Answer_tags Source: Answer_tags]).

[[File:answer_tags_imported.png]]

The flow diagrams below show how controllers in this project are connected with interrelated classes, methods, and views.

[[File:E2169 answer tags controller.png|900px]]

[[File:E2169 tag prompt deployment.png|900px]]

== Testing Plan ==
=== Controller: answer_tags_controller.rb ===

Existing coverage = 0%

Description:
The purpose of this controller is to track down the total number of tags done by the students. The database structure for the answer_tags can be found here [https://expertiza.csc.ncsu.edu/index.php/Answer_tags]

'''Methods'''

1. action_allowed? ( params: action )

Description: To allow the functionality only if the accessing user is having student privileges (current_user_has_student_privileges?)

<code>Use cases:</code>
* <code>When the session is null, Access should be denied</code>
* <code>When the session is of student type, access should be allowed and operation should be executed.</code>

2. index ( params : assignment_id, questionnaire_id, user_id )

Description: Populates the tags by fetching the records from AnswerTag corresponding to each TagPromptDeployment based on the filers provided as params.

<code>Use cases:</code>
* <code> When all valid params are passed, the function should populate the answer tags.</code>
* <code> When user_id is provided, answer tags specific to that user should only be populated.</code>
* <code> When questionnaire_id is provided, answer tags specific to that questionnaire should only be populated.</code>
* <code> When assignment_id is provided, answer tags specific to that assignment should only be populated.</code>
* <code> When only one parameter is passed, the corresponding answer tags should be populated.</code>

<code>Edge test case:</code>
* <code> When user_id is nil, 0 tags prompts should be returned.</code>
* <code> When questionnaire_id is nil, 0 tags prompts should be returned.</code>
* <code> When assignment_id is nil, 0 tags prompts should be returned.</code>

3. create_edit ( params : answer_id, tag_prompt_deployment_id, value )

Description: Fetches the answer tag with the params provided and then updates the respective tag values by the updated values provided as params.

<code>Use cases:</code>
* <code> When the existing tag is found based on the params passed, the existing tag should be updated with new values.</code>

=== Controller: tag_prompt_deployment.rb ===

Existing coverage = 18%

Description:
This Controller is when a user (with instructor privilege) wants to see tagging summary of every user that participated in a tagging assignment.

'''Methods'''

1. tag_prompt
Description: tag_prompt has existing test cases scoring 18 percentage coverage. Additional testing is not required.

2. get_number_of_taggable_answers ( params: user_id)

Description: To calculate total taggable answers reviewed by a user that participated in the given tagging assignment. user_id is the parameter passed to this method. An individual and his team will have the same score. Fig 1: The last column values are processed by this method.

<code>Use cases:</code>
* <code>when a team has given any response then count of taggable answer > 0</code>
* <code>When a question or a response does not exists then count of taggable answer = 0 (extreme case) </code>
* <code>When answer_length_threshold not set, count taggable answer</code>
* <code>When answer_length_threshold is set, count taggable answer</code>

3. assignment_tagging_progress

Description: To compute the number of tagged answers, number of untagged answers, and percentage of answers tagged by the users that participated in the tagging Assignment to a particular question.
Fig 1: First three columns after the 2nd column onwards are updated with results obtained from this method.

<code>Use cases:</code>
* <code>When not team participated in the assignment, assert count(user_answer_tagging) =0</code>
* <code>When the question does not exist, assert count(user_answer_tagging) =0 </code>
* <code>When multiple rounds of review, render ReviewResponseMap and assert response object</code>
* <code>When single round of review, render ResponseMap and assert response object</code>
* <code>When answer_length_threshold is SET, number of answers and when answer_length_threshold is NOT SET, number of answers</code>
* <code>When route is success answer_tagging object structure matching</code>

<code>Edge cases:</code>
* There are no teams associated with the assignment.
* There are no questions in the questionnaire that have the question type specified in the TagPromptDeployment object.
* The assignment is set to vary by round.
* The assignment is set not to vary by round.
* There are multiple tags.

== Bug Fixing of Additional Issues ==

===Model: tag_prompt_deployment.rb (method: get_number_of_taggable_answers)===

An additional issue in the method get_number_of_taggable_answers. The error message is as shown below which indicates string concatenation fails between integer and string data types.

''TypeError:''
no implicit conversion of Fixnum into String at line 23 https://github.com/expertiza/expertiza/blob/beta/app/models/tag_prompt_deployment.rb.

[[File:Get_number_of_taggable_answers_bug.JPG]]

This issue is fixed by correcting the syntax of the two-strings concatenation. After fixing the code is

<code> answers = answers.where(conditions: "length(comments) < #{self.answer_length_threshold}" ) unless self.answer_length_threshold.nil? </code>

===Model: tag_prompt_deployment.rb (method: assignment_tagging_progress)===

<pre>
Failure/Error: answers = answers.where("length(comments) > ?", self.answer_length_threshold.to_s) unless self.answer_length_threshold.nil?

NoMethodError:
undefined method `where' for #<Array:0x00000003424b60>
</pre>

This issue was resolved by using <code>Arrays.select</code> to achieve the same functionality.

There was another small error in which the TeamsUser <code>user</code> was not found, and the resolution was to find the User model for each <code>TeamsUser.user_id</code>.
A similar issue was found with TeamsUser

== Results ==
Test cases are added to cover all functionality. With test cases the new coverage is as follows:

tag_prompt_deployment.rb: Coverage = 99%

answer_tags_controller.rb: Coverage = 100%

Edge cases that are considered in the project are:

-
-
-

== Collaborators ==
Nicholas Himes (Mentor)

1. Aditya Khadse

2. Alec Landow

3. Rageeni Sah

4. Sayali Pranab

== Relevant Links ==
Main Expertiza repository can be found at [https://github.com/expertiza/expertiza/tree/beta GitHubExp]

Our forked repository can be found at [https://github.com/akk5597/expertiza Github]

CSC/ECE 517 Fall 2021

2021-11-06T18:40:13Z

Akkhadse: /* Final Projects */

== OSS Projects ==

* [[CSC/ECE 517 Fall 2021 - E2117. Refactor questionaires_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2128. Refactor student_quizzes_controller.rb & late_policies_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2129. Refactor auth_controller.rb & password_retrieval_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2132. Add tests cases for review mapping helper.rb]]
* [[CSC/ECE 517 Fall 2021 - E2134. Write unit tests for admin_controller.rb and institution_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2138. Auto-generate submission directory names based on assignment]]
* [[CSC/ECE 517 Fall 2021 - E2142. Improve e-mail notifications]]
* [[CSC/ECE 517 Fall 2021 - E2133. Write tests for popup_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2120. Refactor reputation_web_service_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2139. Remove multiple topics at a time]]
* [[CSC/ECE 517 Fall 2021 - E2131. Improve assessment360_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2121. Refactor suggestion_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2122. Refactor impersonate_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2123. Refactor sign_up_sheet_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2126. Refactor account_request_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2124. Refactor review_mapping_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2125. Refactor review_mapping_helper.rb]]
* [[CSC/ECE 517 Fall 2021 - E2127. Refactor teams_controller]]
* [[CSC/ECE 517 Fall 2021 - E2130. Refactor submitted_content_controller.rb]]
* [[CSC/ECE 517 Fall 2021 - E2140. Create new late policy successfully and fix Bank link]]
* [[CSC/ECE 517 Fall 2021 - E2141. OSS project Finklestein: Instructors & Institutions]]
* [[CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task]]
* [[CSC/ECE 517 Fall 2021 - E2147. Role-based reviewing]]
* [[CSC/ECE 517 Fall 2021 - E2145. OSS Project Beige]]
* [[CSC/ECE 517 Fall 2021 - E2146. Introduce a Student View for instructors]]
* [[CSC/ECE 517 Fall 2021 - Refactor Evaluation of SQL Queries]]
* [[CSC/ECE 517 Fall 2021 - E2135. Email notification to reviewers and instructors]]
* [[CSC/ECE 517 Fall 2021 - E2149. Finish Github metrics integration - Reputations]]

== Final Projects ==
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2166._Testing_-_Scoring_%26_Grades#Description_about_project CSC/ECE 517 Fall 2021 - E2166. Testing - Scoring_and_Grades]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2150._Integrate_suggestion_detection_algorithm#Description_about_project CSC/ECE 517 Fall 2021 - E2150. Integrate suggestion detection algorithm]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2151._Allow_reviewers_to_bid_on_what_to_review CSC/ECE 517 Fall 2021 - E2151. Allow reviewers to bid on what to review]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2152._Revision_planning_tool#Description_about_project CSC/ECE 517 Fall 2021 - E2152. Revision_planning_tool]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2170._Testing_-_Response_Maps#Description_about_project CSC/ECE 517 Fall 2021 - E2170. Testing - Response Maps]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2153._Improving_search_facility_in_Expertiza#Description_about_project CSC/ECE 517 Fall 2021 - E2153. Improving search facility in Expertiza]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2162._Further_refactoring_and_improvement_of_review_mapping_helper CSC/ECE 517 Fall 2021 - E2162. Further refactoring and improvement of review mapping helper]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2163._Refactor_waitlist_functionality CSC/ECE 517 Fall 2021 - E2163. Refactor waitlist functionality]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2165._Fix_teammate-review_view CSC/ECE 517 Fall 2021 - E2165. Fix teammate review view ]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2155._Calibration_submissions_should_be_copied_along_with_calibration_assignments CSC/ECE 517 Fall 2021 - E2155. Calibration submissions should be copied along with calibration assignments]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2148._Completion/Progress_View CSC/ECE 517 Fall 2021 - E2148. Completion/Progress view]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2158._Grading_audit_trail CSC/ECE 517 Fall 2021 - E2158. Grading audit trail ]
*[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2159._Expertiza_internationalization CSC/ECE 517 Fall 2021 - E2159. Expertiza internationalization]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2160._Implementing_and_testing_import_export_controllers#Description_about_project CSC/ECE 517 Fall 2021 - E2160. Implementing and testing import and export controllers]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2164._Heatgrid_fixes_and_improvements#Description_about_project CSC/ECE 517 Fall 2021 - E2164. Heatgrid fixes and improvements]
* [[CSC/ECE 517 Fall 2021 - E2168. Testing - Reputations]]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2126._Testing_-_Team_Related_Files#Description_about_project CSC/ECE 517 Fall 2021 - E2126. Testing - Team Related Files]
* [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2021_-_E2156._Issues_related_to_meta-reviewing#Description_about_project CSC/ECE 517 Fall 2021 - E2156. Issues related to meta-reviewing]
* [[CSC/ECE 517 Fall 2021 - E2169. Testing - Answer Tagging]]

CSC/ECE 517 Fall 2021 - E2169. Testing - Answer Tagging

2021-10-31T19:46:37Z

Akkhadse: Initial Setup

CSC/ECE 517 Fall 2021

2021-10-31T19:28:52Z

Akkhadse:

E2144. Refactor delayed mailer and scheduled task

2021-10-31T19:24:39Z

Akkhadse: Akkhadse moved page E2144. Refactor delayed mailer and scheduled task to CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task: Add semester in Title

#REDIRECT [[CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-31T19:24:39Z

Akkhadse: Akkhadse moved page E2144. Refactor delayed mailer and scheduled task to CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task: Add semester in Title

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

If we try to set the due date to be a date that has passed, no new jobs are queued as expected.

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

The following tests were added according to the methods we moved and created:

<pre>
describe "#find_min_from_now_duration" do
it "returns time difference between due_date and now" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 11, 11).in_time_zone)
due_at = Time.parse(DateTime.new(2021, 10, 20, 12, 12, 12).in_time_zone.to_s(:db))
expect(DueDate.find_min_from_now_duration(due_at)).to eq(61)
end
end

describe "#get_dequeue_time_as_seconds_duration_from_now" do
it "returns time difference between now and dequeue of job in seconds" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 00, 00).in_time_zone)
due_at = DateTime.new(2021, 10, 20, 12, 00, 00)
delay_duration = 1.hour
assignment_id = create(:assignment, staggered_deadline: true, name: "testassignment").id
due_date = create(:topic_due_date, deadline_type: @deadline_type,
submission_allowed_id: @deadline_right, review_allowed_id: @deadline_right,
review_of_review_allowed_id: @deadline_right, due_at: due_at, parent_id: assignment_id)
expect(DueDate.get_dequeue_time_as_seconds_duration_from_now(due_date, delay_duration)).to eq(7200)
end
end
</pre>

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, we needed to run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

NB: To create a generic sidekiq worker to test with, you could use
<pre>rails g sidekiq:worker TestWorker</pre>
to create a shell worker class.

===== Sidekiq UI =====
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Ensure the Late Policy is set to <code>--None--</code> under the Due Dates tab. If there is a late policy set, per pre-existing code, the enqueued SimicheckWorker job will be deleted and will not appear in the Sidekiq UI (unless it is dequeued quickly, before the code can delete the jobs in the queue).
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

[[File:simicheck-job-enqueued.png|950px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor.
# Navigate to '''Manage'''>'''Assignments''' in the navigation.
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings.
# Go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Select a late policy next to '''Apply penalty policy:'''. If a late policy has not been created, press '''New Late Policy''' and follow the steps for creating a late policy.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.
# Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the job to be queued.
# To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq.
# Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following:

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Automated Tests ===

The following tests were added to test the workers:

<pre>
describe "test different workers" do
it "should increase the size of queue by 1 when MailWorker is used" do
Sidekiq::Testing.fake!
MailWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when SimicheckWorker is used" do
Sidekiq::Testing.fake!
SimicheckWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when DropOutstandingReviewsWorker is used" do
Sidekiq::Testing.fake!
DropOutstandingReviewsWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end
end
</pre>

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T02:51:31Z

Akkhadse: /* Considering Edge Cases and Other Possibilities */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

If we try to set the due date to be a date that has passed, no new jobs are queued as expected.

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

The following tests were added according to the methods we moved and created:

<pre>
describe "#find_min_from_now_duration" do
it "returns time difference between due_date and now" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 11, 11).in_time_zone)
due_at = Time.parse(DateTime.new(2021, 10, 20, 12, 12, 12).in_time_zone.to_s(:db))
expect(DueDate.find_min_from_now_duration(due_at)).to eq(61)
end
end

describe "#get_dequeue_time_as_seconds_duration_from_now" do
it "returns time difference between now and dequeue of job in seconds" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 00, 00).in_time_zone)
due_at = DateTime.new(2021, 10, 20, 12, 00, 00)
delay_duration = 1.hour
assignment_id = create(:assignment, staggered_deadline: true, name: "testassignment").id
due_date = create(:topic_due_date, deadline_type: @deadline_type,
submission_allowed_id: @deadline_right, review_allowed_id: @deadline_right,
review_of_review_allowed_id: @deadline_right, due_at: due_at, parent_id: assignment_id)
expect(DueDate.get_dequeue_time_as_seconds_duration_from_now(due_date, delay_duration)).to eq(7200)
end
end
</pre>

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, we needed to run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

NB: To create a generic sidekiq worker to test with, you could use
<pre>rails g sidekiq:worker SimicheckWorker</pre>
to create a shell worker class.

===== Sidekiq UI =====
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Ensure the Late Policy is set to <code>--None--</code> under the Due Dates tab. If there is a late policy set, per pre-existing code, the enqueued SimicheckWorker job will be deleted and will not appear in the Sidekiq UI (unless it is dequeued quickly, before the code can delete the jobs in the queue).
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

[[File:simicheck-job-enqueued.png|950px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Automated Tests ===

The following tests were added to test the workers:

<pre>
describe "test different workers" do
it "should increase the size of queue by 1 when MailWorker is used" do
Sidekiq::Testing.fake!
MailWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when SimicheckWorker is used" do
Sidekiq::Testing.fake!
SimicheckWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when DropOutstandingReviewsWorker is used" do
Sidekiq::Testing.fake!
DropOutstandingReviewsWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end
end
</pre>

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T01:44:21Z

Akkhadse: /* Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

The following tests were added according to the methods we moved and created:

<pre>
describe "#find_min_from_now_duration" do
it "returns time difference between due_date and now" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 11, 11).in_time_zone)
due_at = Time.parse(DateTime.new(2021, 10, 20, 12, 12, 12).in_time_zone.to_s(:db))
expect(DueDate.find_min_from_now_duration(due_at)).to eq(61)
end
end

describe "#get_dequeue_time_as_seconds_duration_from_now" do
it "returns time difference between now and dequeue of job in seconds" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 00, 00).in_time_zone)
due_at = DateTime.new(2021, 10, 20, 12, 00, 00)
delay_duration = 1.hour
assignment_id = create(:assignment, staggered_deadline: true, name: "testassignment").id
due_date = create(:topic_due_date, deadline_type: @deadline_type,
submission_allowed_id: @deadline_right, review_allowed_id: @deadline_right,
review_of_review_allowed_id: @deadline_right, due_at: due_at, parent_id: assignment_id)
expect(DueDate.get_dequeue_time_as_seconds_duration_from_now(due_date, delay_duration)).to eq(7200)
end
end
</pre>

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, we needed to run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

NB: To create a generic sidekiq worker to test with, you could use
<pre>rails g sidekiq:worker SimicheckWorker</pre>
to create a shell worker class.

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

==== Automated Tests ====

The following tests were added to test the workers:

<pre>
describe "test different workers" do
it "should increase the size of queue by 1 when MailWorker is used" do
Sidekiq::Testing.fake!
MailWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when SimicheckWorker is used" do
Sidekiq::Testing.fake!
SimicheckWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when DropOutstandingReviewsWorker is used" do
Sidekiq::Testing.fake!
DropOutstandingReviewsWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end
end
</pre>

=== Sidekiq UI ===
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T01:43:25Z

Akkhadse: /* Update and add unit tests according to the refactoring changes made */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

The following tests were added according to the methods we moved and created:

<pre>
describe "#find_min_from_now_duration" do
it "returns time difference between due_date and now" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 11, 11).in_time_zone)
due_at = Time.parse(DateTime.new(2021, 10, 20, 12, 12, 12).in_time_zone.to_s(:db))
expect(DueDate.find_min_from_now_duration(due_at)).to eq(61)
end
end

describe "#get_dequeue_time_as_seconds_duration_from_now" do
it "returns time difference between now and dequeue of job in seconds" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 00, 00).in_time_zone)
due_at = DateTime.new(2021, 10, 20, 12, 00, 00)
delay_duration = 1.hour
assignment_id = create(:assignment, staggered_deadline: true, name: "testassignment").id
due_date = create(:topic_due_date, deadline_type: @deadline_type,
submission_allowed_id: @deadline_right, review_allowed_id: @deadline_right,
review_of_review_allowed_id: @deadline_right, due_at: due_at, parent_id: assignment_id)
expect(DueDate.get_dequeue_time_as_seconds_duration_from_now(due_date, delay_duration)).to eq(7200)
end
end
</pre>

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, we needed to run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

NB: To create a generic sidekiq worker to test with, you could use
<pre>rails g sidekiq:worker SimicheckWorker</pre>
to create a shell worker class.

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Sidekiq UI ===
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T01:36:19Z

Akkhadse: /* Update and add unit tests according to the refactoring changes made */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

The following tests were added:

<pre>
describe "test different workers" do
it "should increase the size of queue by 1 when MailWorker is used" do
Sidekiq::Testing.fake!
MailWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when SimicheckWorker is used" do
Sidekiq::Testing.fake!
SimicheckWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when DropOutstandingReviewsWorker is used" do
Sidekiq::Testing.fake!
DropOutstandingReviewsWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end
end
</pre>

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, a few steps were needed to set up the workers and run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# Initialize three new workers, each having one of the three new worker types:
## <code>rails g sidekiq:worker SimicheckWorker</code>
## <code>rails g sidekiq:worker DropOutstandingReviewsWorker</code>
## <code>rails g sidekiq:worker MailWorker</code>
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Sidekiq UI ===
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T01:34:47Z

Akkhadse: /* Unit Tests */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file: one each for MailWorker, SimicheckWorker, DropOutstandingReviewsWorker.

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, a few steps were needed to set up the workers and run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# Initialize three new workers, each having one of the three new worker types:
## <code>rails g sidekiq:worker SimicheckWorker</code>
## <code>rails g sidekiq:worker DropOutstandingReviewsWorker</code>
## <code>rails g sidekiq:worker MailWorker</code>
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Sidekiq UI ===
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-27T01:32:40Z

Akkhadse: /* Unit Tests */

__TOC__

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.

=== Problem Statement ===

This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq's queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method perform() to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.
The project was partially merged.

What needs to be done:
Feedback from the previous team (E1835) is:
Code in pull request seems pretty reasonable. Some concerns wrt. naming functions, but otherwise reviewers seem to agree that the code is good.

Your pull request has many duplicated commit messages. Please try to <code>squash</code> similar commits.
And using meaningful commit messages later.

You should commit changes to the DB schema (<code>db/schema.rb</code>) only if you have created new DB migrations.
Please double check your code. If you did not aim to change the DB, please revert the DB schema changes. ***** Important, try not to change the DB.

You should not change <code>rails_helper.rb</code> or <code>spec_helper.rb</code> file; please revert these changes.

The previous team’s code needs a refactor in some places, though the actual function of it seems solid. It is very possible that in the three years since this code has been worked on, more mailer functionality has been added. There are other teams this semester working on mailer code as well. Try to keep your code as contained and simple as possible, so that merging these projects is possible.

== Summary of Work Needed ==
After discussing with our project mentor and Dr. Gehringer, we determined the main tasks to be the following:
# Refactor the single Sidekiq worker into multiple workers
# Remove all code related to the <code>drop_one_member_topics</code> deadline type
# Update and add unit tests according to the above changes
# Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected
# Code cleanup

=== Refactor the single Sidekiq worker into multiple workers ===
Previously there was only a single worker class that handled all types of jobs. Its top-level method <code>perform</code> looked like the following.
<pre>
def perform(assignment_id, deadline_type, due_at)
self.assignment_id = assignment_id
self.deadline_type = deadline_type
self.due_at = due_at

assignment = Assignment.find(self.assignment_id)
participant_mails = find_participant_emails

if %w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck].include?(self.deadline_type)
drop_one_member_topics if self.deadline_type == "drop_outstanding_reviews" && assignment.team_assignment
drop_outstanding_reviews if self.deadline_type == "drop_outstanding_reviews"
perform_simicheck_comparisons(self.assignment_id) if self.deadline_type == "compare_files_with_simicheck"
else
# Can we rename deadline_type(metareview) to "teammate review". If, yes then we donot need this if clause below!
deadlineText = if self.deadline_type == "metareview"
"teammate review"
else
self.deadline_type
end

email_reminder(participant_mails, deadlineText) unless participant_mails.empty?
end
end
</pre>

The processing was different depending on the <code>deadline_type</code>. The <code>deadline_type</code>s in
<pre>
%w[drop_one_member_topics drop_outstanding_reviews compare_files_with_simicheck]
</pre>
did not lead to an email reminder being sent, but they were still handled in this <code>MailWorker</code> class.

To make the code more manageable and easier to follow, it made sense to break up the responsibilities of this single <code>MailWorker</code> class into multiple workers:
* <code>SimicheckWorker</code>
* <code>DropOutstandingReviewsWorker</code>
* <code>MailWorker</code>
All three workers extended an "abstract class" called <code>Worker</code> that contained the Sidekiq-related <code>include</code> and the queue name to which all jobs would be sent.

For simplicity, we still had all new workers continue to use a single queue, which we renamed from <code>mailers</code> to <code>jobs</code>. A future improvement could divide these jobs to go to multiple different queues if needed.

=== Remove all code related to the <code>drop_one_member_topics</code> deadline type ===
It was decided that code used for dropping one member or two member teams from topics would never need to be implemented in the Expertiza website. Thus, we removed all code related to the drop_one_member_topics method within the Expertiza beta. This involved removing the <code>drop_one_member_topics</code> function from <code>drop_outstanding_reviews_mail_worker.rb</code>, now renamed <code>drop_outstanding_reviews_worker.rb</code>. The <code>drop_one_member_topics</code> method was called from the <code>prepare_data</code> method, which was then called by the Sidekiq method <code>perform</code> so we removed this call. The <code>drop_one_member_topics</code> was also referenced in <code>assignment_form.rb</code> using <code>add_delayed_job</code> in the <code>add_to_delayed_queue</code> method and in <code>delayed_mailer.html.erb</code> as an option for the handler so both of these code snippets were removed as well.

=== Code Cleanup ===
Smaller improvements done while implementing the main tasks included:
* Using ActiveSupport::Duration objects in order to calculate the minutes and seconds related to the dequeue times
* Refactoring the time calculations for the dequeue times to the DueDate class
* Correcting the parameters passed to the workers, which were previously incorrect. As an example:
<blockquote>
<pre>
def add_delayed_job(_assignment, deadline_type, due_date, min_left)
delayed_job_id = MailWorker.perform_in(min_left * 60, due_date.parent_id, deadline_type, due_date.due_at)
delayed_job_id
end
</pre>
The first parameter after the time delay must be assignment ID, and <code>due_date.parent_id</code> is not an assignment ID.
</blockquote>
* Removing whitespace issues reported by codeclimate
* Refactoring all variable names to be in snake_case
* Removing identical blocks of code
* Adding method comments to make methods more understandable

== Test Plan ==

=== Unit Tests ===
In order to properly test that our refactoring had worked properly, we found that we would have to add in multiple tests to account for our new methods as well as refactor old tests to work with the changes that we had made.

For every worker, we have added a test case in the <code>spec/workers/sidekiq_mail_worker_spec.rb</code> file:

<pre>
describe "test different workers" do
it "should increase the size of queue by 1 when MailWorker is used" do
Sidekiq::Testing.fake!
MailWorker.perform_in(42.minutes, 1, "review", "2021-10-27 00:00:01")
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when SimicheckWorker is used" do
Sidekiq::Testing.fake!
SimicheckWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end

it "should increase the size of queue by 1 when DropOutstandingReviewsWorker is used" do
Sidekiq::Testing.fake!
DropOutstandingReviewsWorker.perform_in(42.minutes, 1)
queue = Sidekiq::Queues["jobs"]
expect(queue.size).to eq(1)
queue.clear
end
end
</pre>

Further, we moved methods <code>find_min_from_now_duration</code> from <code>assignment_form.rb</code> to <code>due_date.rb</code>. Along with that we introduced some methods to make it easier to understand the conversions and values that are being set.

<pre>
describe "#find_min_from_now_duration" do
it "returns time difference between due_date and now" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 11, 11).in_time_zone)
due_at = Time.parse(DateTime.new(2021, 10, 20, 12, 12, 12).in_time_zone.to_s(:db))
expect(DueDate.find_min_from_now_duration(due_at)).to eq(61)
end
end

describe "#get_dequeue_time_as_seconds_duration_from_now" do
it "returns time difference between now and dequeue of job in seconds" do
allow(DateTime).to receive(:now).and_return(DateTime.new(2021, 10, 20, 11, 00, 00).in_time_zone)
due_at = DateTime.new(2021, 10, 20, 12, 00, 00)
delay_duration = 1.hour
assignment_id = create(:assignment, staggered_deadline: true, name: "testassignment").id
due_date = create(:topic_due_date, deadline_type: @deadline_type,
submission_allowed_id: @deadline_right, review_allowed_id: @deadline_right,
review_of_review_allowed_id: @deadline_right, due_at: due_at, parent_id: assignment_id)
expect(DueDate.get_dequeue_time_as_seconds_duration_from_now(due_date, delay_duration)).to eq(7200)
end
end
</pre>

=== Functional Tests ===
We decided that functional testing would need to be done by manually checking to see if Sidekiq tasks were queued as expected when changes to assignments were made. The three important functional tests that we decided needed to be verified were that the Simicheck Worker was queuing, the Mail Worker was queueing, and the Drop Outstanding Reviews Worker was queuing. These could be considered verified under the condition that the '''When''' field, '''Job''' field, and '''Arguments''' field in the Sidekiq view all came out as expected.

=== Considering Edge Cases and Other Possibilities ===

== Implementing Test Plan ==

=== Update and add unit tests according to the refactoring changes made ===

=== Test all new worker classes to ensure Sidekiq jobs were being dequeued and run as expected ===
==== Sidekiq setup ====
In order to test the Sidekiq workers, a few steps were needed to set up the workers and run Sidekiq (steps were obtained from [https://tudip.com/blog-post/how-to-use-sidekiq-in-rails-for-background-processing/ this site]):
# Initialize three new workers, each having one of the three new worker types:
## <code>rails g sidekiq:worker SimicheckWorker</code>
## <code>rails g sidekiq:worker DropOutstandingReviewsWorker</code>
## <code>rails g sidekiq:worker MailWorker</code>
# In a terminal separate from the one in which the main rails application is running, run
<blockquote>
<pre>
bundle exec sidekiq
</pre>
</blockquote>

This Sidekiq-specific terminal will show logs and printed statements from the workers.

==== SimicheckWorker Test ====
To enqueue a Simicheck job:
# Edit an assignment.
# Select the <code>Use simicheck?</code> box.
# Set <code>Simicheck Delay</code> to <code>0</code> (or to a nonzero value to observe a delay).
# Set the SimiCheck Similarity Threshold to a nonzero value.
# Click <code>Save</code>.
# Observe a Simicheck Worker job enqueued and dequeued in the Sidekiq UI.

[[File:Enqueue-simicheck-job.png|500px|How to Enqueue a Simicheck Worker Job]]

==== MailWorker and DropOustandingReviews Test ====
The Mail Worker sends out notifications to students if the due date of an assignment is changed. Drop Outstanding Reviews Worker drops all reviews that had not been worked on before the due date was changed. These both can be tested together, as when an assignment with a review is updated, Sidekiq should queue both the jobs for Mail Worker and Drop Outstanding Reviews Worker. To test these:
# Log into Expertiza as an instructor
# Navigate to '''Manage'''>'''Assignments''' in the navigation
# Click the plus button in the top right corner on the '''Manage Assignments''' and create an assignment with default settings. Make sure when creating the assignment to go to the '''Due Dates''' tab and assign due dates for both the assignment and the review.
# Assign student7613 to the assignment you created by clicking the person with the plus icon in the icons next to the assignment on the '''Manage Assignments''' page.

At this point, we realized we need to alter the code to test if the code was properly queuing the Mail Worker because the current if statement to add items to the delayed queue required that it have a late policy. However, the late policy code is not currently working. Thus, in the <code>assignment_form.rb</code> model file, we edited the following code:

[[File:Change_late_policy.png|950px|Late policy code to alter]]

We removed <code>and has_late_policy</code> so <code>add_delayed_queue</code> can run even without late policies being implemented. After saving this file, we started up Expertiza again.
<ol start="5">
<li> Go to the assignment that student7613 had been assigned to </li>
<li> Edit the assignment, go to the due dates tab, and change the due date on the review to be a day ahead of its current due date, and press save. The assignment due date must be changed to a time at least an hour ahead of the current time for the reminder to be sent. </li>
<li> To ensure that the assignments are now queued on Sidekiq, visit the Sidekiq UI by going to [Expertiza VCL URL]/sidekiq. </li>
<li> Upon arriving on this page, verify that the jobs added to the queue were correct by clicking to '''Scheduled'''. After arriving on this page, you should see something like the following: </li>
</ol>

[[File:Sidekiq-jobs.png|950px|Sidekiq jobs]]

The job '''MailWorker''' that had been queued successfully shows that our refactoring of Mail Worker had worked properly and the Mail Worker was operational. The job '''DropOutstandingReviewsWorker''' showed that our refactoring of Drop Outstanding Reviews Worker had also been done correctly.

=== Sidekiq UI ===
To see which messages were being enqueued and dequeued in Sidekiq, we use the Sidekiq user interface, which can be accessed via the <code>/sidekiq</code> route of the application.
[[File:Sidekiq-ui.png|950px|The Sidekiq UI]]

CSC/ECE 517 Fall 2021 - E2144. Refactor delayed mailer and scheduled task

2021-10-06T15:59:01Z

Akkhadse: Wiki page for E2144

== Background ==
This project revolves around the Sidekiq gem for asynchronous processing of email tasks. It has a queue system to hold and then process jobs. Sidekiq’s queue replaces DelayedMailer’s queue. The previous team that worked on this also created a method <code>perform()</code> to gather email IDs of all participants in an assignment and send them an email reminder. Some test cases exist for this work.

=== Previous Setup ===
Initial work can be found [https://expertiza.csc.ncsu.edu/index.php/E1835_Refactor_delayed_mailer_and_scheduled_task here].

Code in the [https://github.com/expertiza/expertiza/pull/1245/files pull request] was found to be reasonable and working. The method <code>perform()</code> has:
{| class=wikitable style=text-align:right
|+ Metrics for <code>perform</code> method
|-
! Metric !! Current Value !! Limit
|-
! Cognitive Complexity
| 15 || 5
|-
! Cyclomatic Complexity
| 8 || 6
|-
! Assignment Branch Condition size
| 22.93 || 15
|-
! Perceived complexity
| 10 || 7
|}

Previous pull request also has duplicated commit messages and the commits need to be squashed.

Files involved (some changed should not have been changed):
* app/controllers/assignments_controller.rb
* app/mailers/mail_worker.rb
* app/models/assignment_form.rb
* config/sidekiq.yml
* spec/models/assignment_form_spec.rb
* spec/sidekiq_mail_worker_spec.rb
* spec/spec_helper.rb

There are minor styling issues such as missing or trailing whitespaces, lines that are too long, extra empty line, unused arguments.