CSC/ECE 517 Fall 2019 - E1995. Tests for email functionality

Expertiza Background

Expertiza is an educational web application created and maintained by the joint efforts of the students and the faculty at NCSU. It’s an open source project developed on Ruby on Rails platform and the code is available on Github. It allows students to review each other’s work and improve their work upon this feedback. Students and Instructors (including TA's) use this application though their credentials. Instructors can add new projects, assignments etc as well edit the previous ones and at a later stage can see the student submission and grade them. They can even put a time limit (deadline) for submitting the assignment. Students can form teams for the projects, submit their work through handles (Wiki page, link to a video etc). Students can even review their teammates and other peers, as well give them any suggestions if required

Problem Statement

Expertiza system requires sending emails to students and instructors on occurrence of various events or to facilitate actions further.

Email notifications are sent out when the following events occur:

1. An account holder is emailed when a new account is created.
2. The author of a submission gets notified via email when a reviewer has reviewed their work and the feedback is available to view.
3. The reviewer receives an email when work has been submitted/resubmitted by an author whose work needs to reviewed.
4. A confirmation email is sent to a user of the system when they reset their Expertiza account password
5. An instructor may configure a setting for an assignment where an email is sent to all the participants to remind them of an upcoming deadline.

In the current scenario, it has been identified that not all emailing functionalities are working as desired. Also, presently, there is little evidence to claim that the existing implementation is covering all cases and handling all necessary conditions.

Our project is to build robust test suite for mailer functionality that is currently implemented in Expertiza. Developing this would ensure that all the mailing features are backed up with a stronger set of test features.

Implementation

Scenario 4

There are many ways to create new user account in Expertiza. Irrespective of the method of creating user, that user should get notified via email whenever an account is created stating this is your username and will also provide randomized password to allow them to login for the first time. Below given are the ways to create new user to system.

1. Instructor can add users to the system by filling the sign-up form for user.

2. Instructor can also import CSV file containing the user information to add user to the system.

3. Instructor can add user as the participant of the assignment by importing details through CSV file. While doing that, if there are any user credentials present in the CSV file but is not registered in the system, then in that case new user account will be created.

4. There is also way to request user account without any help of the instructor. In that case, actual account will not be created but the details will be stored in the requested users table. Whenever instructor authorizes the requested user, actual user account will be created, and the user of that account should receive email notification stating your account request has been approved.

Cases Tested

There are 3 ways in which user can be created, which are implemented as 3 different scenarios
i) instructor filling up the form
ii) User requests account
iii) by importing CSV file from user page  :

The following features were checked as part of user creating user in scenario i and scenario ii
1. Mail delivery count increased by 1 after the mail triggered by the scenario is sent
2. Sender email-id is verified. The sender mail-id is verified to be 'expertiza.development@gmail.com'
3. Receiver email id is checked to be equal to 'expertiza.development@gmail.com' . Since all mails triggered have a record in 'expertiza.development@gmail.com' when executed in test environment.
4. The subject is checked to be equal to 'Your Expertiza account and password has been created'

We considered the following cases as it blocks while testing specific to scenario 2:
1. no email should be sent if new account requested with invalid email address
2. super admin should receive email about new account request with a valid email address
3. email notification should be sent to requester when super admin approves account creation request
4. email notification should be sent to requester when super admin rejects account creation request

We implemented scenario iii using rspec unit tests:
1. 2. 3. 4.

Scenario 5

When the user tries to login, and the password entered is wrong, the user is redirected to reset password page.

Here the email-id entered is searched through the database, If such user account is found: url for reset is generated, with random token set as password is sent to the user.

Files associated with this functionality:

1. spec/controllers/PasswordRetrievalController
2. app/controllers/PasswordRetrievalController
3. app/mailers/mailer.rb
4. app/helper/mailer_helper.rb

Flowchart:

Cases Tested

The following cases were considered as 'context blocks' while testing the mailer function when user requests for password:
1. user enters email id which is not associated with any account
2. user enters email id that's present in the user database

We used Capybara to model the scenario with following steps:
1. visit expertiza home page
2. click 'Forgot password?' link
3. after rediection to password retrieval page
4. fill in user email field with 'expertiza@mailinator.com'
5. click on 'Request password' button

Outcome

We used Capybara to model the scenario with following steps:
1. visit expertiza home page
2. click 'Forgot password?' link
3. after rediection to password retrieval page
4. fill in user email field with 'expertiza@mailinator.com'
5. click on 'Request password' button

-- result x test cases passed image --

Team

- - - -

References

Expertiza on GitHub

GitHub Pull Request

demo link

To visit our forked repo, click here

View our pull request here

Proposed Approach

We propose to reach our goal in increments of the steps listed below:

• Identify the various scenarios where email functionality is being called
• Check if the email functionality is presently working for each scenario
• Ensure necessary parameters are being passed and received
• Verify if email is being sent out to correct recipient
• Check if email is being sent out when expected to (some functionalities require the email to be sent real-time, as a trigger on an action while some require to monitor activity/follow a pre-defined schedule)

On having fair idea of the all of the above, we aim on building test cases that will perform all these checks automatically, in real time and also test how well these work in conjunction with all other parts of the application as a whole.

The proposed test-suite is a combination of unit tests and integration tests that will check the control flow and correctness concerning emails within the system.

Test Plan

Automated Testing using Rspec

Listed below are a few scenarios we have identified so far, which need to be tested for correct functioning

Scenario 1: Deadline reminder emails

The system continuously monitors upcoming deadlines of assignments. If an assignment is configured such that an email reminder must be sent for an approaching deadline, then an email must be sent to all participants of that assignment as scheduled by the instructor.

Files associated with this functionality:

1. mail_worker.rb
2. background_email_reminder.rake

Following tests will be performed to verify whether the functionality is working as desired.

• Check if email notification is being sent to all the participants on the assignment. This can be verified by confirming that the count of emails sent out is equal to the total number of student participants on that assignment
• Check if the deadline type is correct and relevant

Scenario 2: Author(s) get emailed when a reviewer has reviewed their work

Whenever someone reviews other student or team’s work, then that student or all the members of that team should get notified through the email stating that new peer review is available for your work. Even when the reviewer updates the reviews, then also that student or all the team members should get notified that updated comments are present on their submitted work.

Files associated with this functionality:

1. app/controllers/response_controller.rb
2. app/models/review_response_map.rb

Following parameters need to be tested:

• Whenever reviewer reviews some team’s work, in that case all the team members should receive an email. So, number of emails delivered should be number of participants assigned to that work.

Scenario 3: Reviewer gets notified when an author has submitted/resubmitted work

This will be mainly the case when there will be more than one round of the submission. After the first submission, some users will be assigned to review the submitted work. Now, in second round of the submission, whenever author resubmits or edit their work, then in that case all the reviewers assigned to that author should get email notification stating new submission is available for them to review.

Files associated with this functionality:

1. app/controllers/submitted_content_controller.rb
2. app/helpers/mailer_helper.rb

Following parameters need to be tested:

• Whenever the team or students resubmits their work, all the reviewers assigned to them should get notified. That means number of mails delivered should be equals to the number of reviews assigned to their work.

Scenario 4: User gets an account creation notification upon account creation with that email id

There are many ways to create new user account in Expertiza. Irrespective of the method of creating user, that user should get notified via email whenever an account is created stating this is your username and will also provide randomized password to allow them to login for the first time. Below given are the ways to create new user to system.

1. Instructor can add users to the system by filling the sign-up form for user.

2. Instructor can also import CSV file containing the user information to add user to the system.

3. Instructor can add user as the participant of the assignment by importing details through CSV file. While doing that, if there are any user credentials present in the CSV file but is not registered in the system, then in that case new user account will be created.

4. There is also way to request user account without any help of the instructor. In that case, actual account will not be created but the details will be stored in the requested users table. Whenever instructor authorizes the requested user, actual user account will be created, and the user of that account should receive email notification stating your account request has been approved.

Files associated with this functionality:

1. app/mailers/mailer.rb
2. app/helper/mailer_helper.rb
3. app/models/user.rb
4. app/models/assignment_participant.rb
5. app/controllers/user_controller.rb

Following parameters must be tested

• Check if the email is being sent to the same user whose account has been newly created
• Verify that only one email is being sent for each corresponding new account

Scenario 5: User gets password reset link, on request for new password

When the user tries to login, and the password entered is wrong, the user is redirected to reset password page.

Here the email-id entered is searched through the database, If such user account is found: url for reset is generated, with random token set as password is sent to the user.

Files associated with this functionality:

1. spec/controllers/PasswordRetrievalController
2. app/controllers/PasswordRetrievalController
3. app/mailers/mailer.rb
4. app/helper/mailer_helper.rb

Flowchart:

Following tests can be performed to verify whether the functionality is working as desired. When a new user is created :

• Expect mail to be generated and sent to the mail id of the user

• Expect the subject of mail to be "Expertiza password reset"

• The link in email body with token generated in send_password can be checked

Results

• 162 issues resolved
• 42 Files changed
• 215 additions
• 163 Deletions
• Travis CI build passed
• No merge conflicts
• coveralls :

       *  Coverage increased (+10.1%) to 50.97%.

• codeclimate bot

       * Code Climate has analyzed commit 050fa7d and detected 0 issues on this pull request.
       * The test coverage on the diff in this pull request is 42.3% (50% is the threshold).
       * This pull request will bring the total coverage in the repository to 29.0% (-21.9% change).

• travis-ci

       * Your changes look good to me! tada
       * The Travis CI build passed

Implementation

Fixes:

--the additional fixes--

Tests:

The Capybara tests were written to model the interactions of the user on each of the specific scenarios. Rspec scripts were written to test if the mail has been sent

User account creation (Scenario 4) Implementation

There are 3 ways in which user can be created, which are implemented as 3 different scenarios
i) instructor filling up the form
ii) User requests account
iii) by importing CSV file from user page  :

The following features were checked as part of user creating user through all methods
1. Mail delivery count increased by 1 after the mail triggered by the scenario is sent
2. Sender email-id is verified. The sender mail-id is verified to be 'expertiza.development@gmail.com'
3. Receiver email id is checked to be equal to 'expertiza.development@gmail.com' . Since all mails triggered have a record in 'expertiza.development@gmail.com' when executed in test environment.
4. The subject is checked to be equal to 'Your Expertiza account and password has been created'

We considered the following cases as it blocks while testing specific to scenario 2:
1. no email should be sent if new account requested with invalid email address
2. super admin should receive email about new account request with a valid email address
3. email notification should be sent to requester when super admin approves account creation request
4. email notification should be sent to requester when super admin rejects account creation request

We used Capybara to model the scenario as follows:
i) create user through instructor filling up the form:
Admin user is created using factory class.
1.visit expertiza
2.login_as 'admin_user'
3.visit list of users
4.click New User link
5.fill user_name with 'teststudent'
6.fill user_fullname with 'test student'
7.fill user_email with 'test@student.com'
8.check checkbox to send email to user on review
9.click 'Create' button.

ii) create user through instructor filling up the form:
super admin is created using factory class.
1.visit expertiza
2.click Request Account link
3.select instructor from dropdown
4.fill user name with 'requester1'
5.fill user fullname with 'requester1, requester1'
6.fill user email with 'test.com'
7.fill user_email with 'test@student.com'
8.select 'North Carolina State University' from user institution
9.click 'Create' button.
10.visit list of pending requests from users
11.Requested user's status is first 'Under Review'
12.Set status as Approved or Rejected
13.click on 'Submit' button

Password Reset (Scenario 5) Implementation:

The following features were checked inside 'it blocks' as part of verifying email functionality
1. Mail delivery count increased by 1 after the mail triggered by the scenario is sent
2. The sender mail-id is verified to be 'expertiza.development@gmail.com'
3. Receiver email id is checked to be equal to 'expertiza.development@gmail.com' . Since all mails triggered have a record in 'expertiza.development@gmail.com' when executed in test environment.
4. The subject is checked to be equal to 'Expertiza password reset'

The following cases were considered as 'context blocks' while testing the mailer function when user requests for password:
1. user enters email id which is not associated with any account
2. user enters email id that's present in the user database

We used Capybara to model the scenario with following steps:
1. visit expertiza home page
2. click 'Forgot password?' link
3. after rediection to password retrieval page
4. fill in user email field with 'expertiza@mailinator.com'
5. click on 'Request password' button

Quick Links

To visit our forked repo, click here

View our pull request here