CSC/ECE 517 Spring 2023 - E2302
E2302. Refactoring the Delayed Mailer and Scheduler
Expertiza Overview
Background
Expertiza is an open-source software written using Ruby on Rails which functions as a learning management software system. It has many different functions and abilities including the ability to create assignments, quizzes, assignment groups, and topics, and also a complete mechanism for providing peer reviews and feedback for other groups and other teammates.
Sidekiq is a popular open-source gem for Ruby on Rails applications that provides a simple and efficient way to perform background processing for time-consuming tasks. It uses multi-threading to handle jobs asynchronously, allowing the main application to continue running while the background jobs are executed. Sidekiq provides a queue system that manages jobs and prioritizes them based on their urgency. The docs can be found here [1].
Sidekiq is used in Expertiza to schedule background tasks such as Plagiarism checks for Assignments (Similink) and the mail for assignment's due dates.
Based on our task prompt we understood that the create method of the assignment controller needs to be changed alongside the testing needs to be improved for sidekiq-related worker files. Also, the code should be commented more thoroughly and variable names changed to better reflect the context they are used for.
Prior Work
History: Previous projects
* E2253 [2] * E2144 [3] * E2273 [4]
The latest Pull Request was not merged into the repository due to some failing tests and missing comments.
Tests were failing in the following files-
spec/models/assignment_form_spec.rbspec/controllers/assignments_controller_spec.rb
We try to address those failing tests and add block comments before function definition wherever needed.
Additional testing of some related files is also in the scope of the project.
Our Changes
Upon review of previous PRs, we found that the work was thorough, albeit a few test cases were failing, and comments and methods could be refactored better. Based on the prompts we got, we focussed on refactoring the create method of assignments controller, and assignments form, along with the refactoring of sidekiq-related files.
Below is the summary of all the changes as part of this PR.
Changes to spec/controllers/auth_controller_spec.rb
| # | Change | Rationale | Commit Link |
|---|---|---|---|
| 1 | Refactored the create method into multiple methods
|
The current create method is long and have multiple complex references and if statements. Created separate helpers to simplify complex DB context
|
Commit |
| 2 | Used early return in case of errors | We refactored the code such that the errors like the assignment already being present or the directory already being present were caught early in the code. It leads to cleaner code as well. | Commit |
| 3 | Renamed variable names | Some of the current variable names were ambiguous as to what they meant, (cur_questionnaire, cur_due, etc.). Renamed those names to be less ambiguous and state their function.
|
Commit |
| 4 | Added comments to other helper methods | Some functions had no comments which were needed. | Commit |
Changes to app/mailers/mail_worker.rb
| # | Change | Rationale | Commit Link |
|---|---|---|---|
| 1 | Removed inline comments and added block comments to the function perform | Block comments before the function allows better readability and looks clean | Commit |
| 2 | Moved the line to find the participants of a course inside the conditional block | Earlier the list of participants of a course were found even when the deadline_type was 'drop_outstanding_reviews’. We do find the list of emails when it is not 'drop_outstanding_reviews’ and send emails to them.
|
Commit |
| 3 | Combined the multiple attr_accessorlines into a single line
|
Better readability and shorter function code. | Commit |
| 4 | @mail object is not used as it is not reused anywhere else | Any temporary variable that is used once should be avoided as it takes up valuable resources. | Commit |
Changes to spec/models/assignment_form_spec.rb
| # | Change | Rationale | Commit Link |
|---|---|---|---|
| 1 | Replaced ‘mailers’ with ‘jobs’ in add_to_delayed_queue test
|
Test was failing due to checking the wrong Sidekiq queue, currently add_to_delayed_queue adds to job queue NOT mailers queue
|
Commit |
Changes to app/models/assignment_form.rb
| # | Change | Rationale | Commit Link |
|---|---|---|---|
| 1 | Added block comments to methods and removed inline comments | Better readability and looks clean | Commit |
| 2 | Created default_assignment_questionaire to handle creation of a default assignment_questionnaire that was originally in assignment questionnaire
|
assignment_questionnaire originally not only searched for an assignment_questionnaire but created one if one matching type did not exist. Now the second functionality (creating a default assignment_questionnaire) is its own method.
|
Commit |
| 3 | In line 58 of original code replaced [][] method of searching with .dig
|
This particular line could throw errors and .dig is a safer way to search a table without the risk of getting nil errors | Commit |
Changes to spec/sidekiq_mail_worker_spec.rb
| # | Change | Rationale | Commit Link |
|---|---|---|---|
| 1 | Added test case for method perform in mail_worker.rb
|
This test case was not implemented in the earlier Pull Request | Commit |
| 2 | Added test case for method find_participant_emails in mail_worker.rb
|
This test case was not implemented in the earlier Pull Request | Commit |
| 3 | Added test case for method email_reminder in mail_worker.rb
|
This test case was not implemented in the earlier Pull Request | Commit |
spec/models/assignment_form_spec.rb
After making the changes to assignment_form.rb and assignment_form_spec.rb we ran the rspec tests with the command bundle exec rspec spec/models/assignment_form_spec.rb
spec/workers/sidekiq_mail_worker_spec.rb
Automated testing of sidekiq_mailer_worker_spec.rb with the command bundle exec rspec spec/workers/sidekiq_mailer_worker_spec.rb
spec/controllers/auth_controller_spec.rb
Automated testing of assignments_controller_spec.rb with the command bundle exec rspec spec/controllers/assignment_form_spec.rb
Testing / Test Plan
The enhancement is tested manually and through rspec tests.
VCL : http://152.7.178.223:8080/
Login: instructor_Lien
Password: password
To Test via UI:
1) Login to VCL with the above credentials
2) Click on Manage tab
3) Click on Assignments
4) click on + to add a new assignment
5) Create an assignment with a non-zero delay(this feature is to send a reminder email to participants of the course) in SimiCheck(this is the Plagiarism Checker that is being used) and Similarity Threshold to a non-zero.
6) Go to the due date and add a due date for any day after the current date and timestamp. Just be sure that you have atleast 'delay'(set above while creating the assignment) number of hours left for the deadline. Or else the jobs would not be scheduled properly.
7) Create the assignment
8) Go to the Assignment page and add participants (ex. 'Paul' & 'Dan' to name a few. You can query the database and find more Students in the test database) to the assignment.
9) Check the sidekiq dashboard [5] page for queued jobs. Check the time left for the MailWorker that would be sending out the mails.
10) It would be under the scheduled tab and would have 'delay' number of hours left less that the actual time left for the deadline. This indicated that the Mailer jobs is queued in a perfect way to send out reminder emails.
- Edge Cases **
Error shown in the dashboard:
1) Note: In previous refactoring changes it was noted that this error is to be expected and will not affect the real Expertiza. Error is due to Simicheck Plagiarism Checker not having access to certain features in the PR build. This is currently out of the scope of the project and is yet to be implemented in totality. Also the Scheduled MailWorker that sends the email out to user would fail because of lack of SMTP authentication in the test database and environment. It is present in the production environment to which we dont have access.
2) If setting up the environment yourself, the users table in expertiza_development DB would need to be updated and a Preferred Time needed to be added in already created users, for assignments to be saved.