Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2021 - E2168. Testing - Reputations

2021-11-30T03:28:06Z

Dli35: /* send_post_request */

== Project Overview ==

=== Introduction ===

<q>Online peer-review systems are now in common use in higher education. They free the instructor and course staff from having to provide personally all the feedback that students receive on their work. However, if we want to assure that all students receive competent feedback, or even use peer-assigned grades, we need a way to judge which peer reviewers are most credible. The solution is to use a reputation system.</q> The reputation system is meant to provide objective value to student assigned peer review scores. Students select from a list of tasks to be performed and then prepare their work and submit it to a peer-review system. The work is then reviewed by other students who offer comments/graded feedback to help the submitters improve their work.
During the peer review period it is important to determine which reviews are more accurate and show higher quality. Reputation is one way to achieve this goal; it is a quantization measurement to judge which peer reviewers are more reliable.
Peer reviewers can use expertiza to score an author. If Expertiza shows a confidence ratings for grades based upon the reviewers reputation then authors can more easily determine the legitimacy of the peer assigned score. In addition, the teaching staff can examine the quality of each peer review based on reputation values and, potentially, crowd-source a significant portion of the grading function.
Currently the reputation system is implemented in Expertiza through web-service, but there's no test written for it. Thus our goal is to set up assignments and reviews that would produce specific reputation scores, and test that the correct reputations are in fact being produced.

=== System Design ===
The below is referenced from project E1625, which would give us the overall description of the reputation system. 
[[File:Diagram E1625.png|1000px]] 
There are two algorithms intended for use in calculation of the reputation values for participants.

There is a [https://expertiza.ncsu.edu/reputation_web_service/client web-service] (the link accessible only to the instructors) available which serves a JSON response containing the reputation value based on the seed provided in the form of the last known reputation value which we store in the ''participants'' table. An instructor can specify which algorithm to use for a particular assignment to calculate the confidence rating.

As the [https://docs.google.com/viewer?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FYang_Song36%2Fpublication%2F289528736_Pluggable_Reputation_Systems_for_Peer_Review_a_Web-Service_Approach%2Flinks%2F568ec8d008ae78cc05160aed.pdf%3FinViewer%3D0%26pdfJsDownload%3D0%26origin%3Dpublication_detail paper] on reputation system by observes, “the Hamer-peer algorithm has the lowest maximum absolute bias and the Lauw-peer algorithm has the lowest overall bias.This indicates, from the instructor’s perspective, if there are further assignments of this kind, expert grading may not be necessary.”
:;Reputation range of Hamer’s algorithm is :
:red value < 0.5
:yellow value is >= 0.5 and <= 1
:orange value is > 1 and <= 1.5
:light green value is > 1.5 and <= 2
:green value is > 2

 
The main difference between the Hamer-peer and the Lauw-peer algorithm is that the Lauw-peer algorithm keeps track of the reviewer's leniency (“bias”), which can be either positive or negative. A positive leniency indicates the reviewer tends to give higher scores than average. This project determines reputation by subtracting the absolute value of the leniency from 1. Additionally, the range for Hamer’s algorithm is (0,∞) while for Lauw’s algorithm it is [0,1].
:;Reputation range of Lauw’s algorithm is
:red value is < 0.2
:yellow value is >= 0.2 and <= 0.4
:orange value is > 0.4 and <= 0.6
:light green value is > 0.6 and <= 0.8
:green value is > 0.8

The instructor can choose to show results from Hamer’s algorithm or Lauw’s algorithm. The default algorithm should be Lauw’s algorithm.

=== Objectives ===

Our objectives for this project are the following:
* Double and stub an assignment, a few submissions to the assignment, under different review rubrics
* Manually calculate reputation scores based on paper "Pluggable reputation systems for peer review: A web-service approach"
* Validate correct reputation scores based on different review rubrics generated by reputation management VS manual computation of reputation score using the Hamer's and Lauw's algorithm.

=== Files Involved ===
*reputation_web_service_controller_spec

== Test Plan ==

=== Setup Testing Objects ===
In order to implement testing on reputation, it is crucial to create sample reviews so that we could possibly obtain reputation score. During the kickoff meeting, our team defined four necessary steps to follow for the purpose of testing. Also, appropriate objects could be created and confined as discussed below.
 
The manifestation of each object will contribute to the success of the following test on reputations. Some fields in each object can be empty or have default values. Some attributes are not relevant to the test. When implementing the test, the test scripts need to generate or set fixed values for corresponding fields.

==== Assignment ====
* Objects involved 
[[assignments]] 

* Essential Parameters to be configured 
submitter_count = 0; 
num_reviews = 5; 
num_reviewers = 5; 
num_reviews_allowed = 5; 
rounds_of_reviews = 2; 
reputation_algorithm = lauw/hamer; 
Note: Two assignment objects were created. Assignment_1 used lauw's algorithm, whereas assignment_2 used hamer's alogorithm. 

* Code Implemented 
<pre>
@assignment_1 = create(:assignment, created_at: DateTime.now.in_time_zone - 13.day, submitter_count: 0, num_reviews: 3, num_reviewers: 5, num_reviews_allowed: 5, rounds_of_reviews: 2, reputation_algorithm: 'lauw', id: 1)
@assignment_2 = create(:assignment, created_at: DateTime.now.in_time_zone - 13.day, submitter_count: 0, num_reviews: 3, num_reviewers: 5, num_reviews_allowed: 5, rounds_of_reviews: 2, reputation_algorithm: 'hamer', id: 2)
</pre>

==== Questionnaires(Rubrics) ====
* Objects involved 
[[questionnaires]] 
[[assignment_questionnaires]] 

* Essential Parameters to be configured 
min_question_score = 0; 
max_question_score = 5; 
type = ReviewQuestionnaire; 
Note: We will define the assignment with ReviewQuestionnaire type rubric. 

* Code Implemented 
<pre>
@questionnaire_1 = create(:questionnaire, min_question_score: 0, max_question_score: 5, type: 'ReviewQuestionnaire', id: 1)
@assignment_questionnaire_1_1 = create(:assignment_questionnaire, assignment_id: @assignment_1.id, questionnaire_id: @questionnaire_1.id, used_in_round: 1)
@assignment_questionnaire_1_2 = create(:assignment_questionnaire, assignment_id: @assignment_1.id, questionnaire_id: @questionnaire_1.id, used_in_round: 2)
@assignment_questionnaire_2_1 = create(:assignment_questionnaire, assignment_id: @assignment_2.id, questionnaire_id: @questionnaire_1.id, used_in_round: 1)
@assignment_questionnaire_2_2 = create(:assignment_questionnaire, assignment_id: @assignment_2.id, questionnaire_id: @questionnaire_1.id, used_in_round: 2)
</pre>

==== Questions under Questionnaires ====
* Objects involved 
[[questions]] 

* Code Implemented 
<pre>
# question_j_j means question #j in questionnaire #i.
@question_1_1 = create(:question, questionnaire_id: @questionnaire_1.id, id: 1)
@question_1_2 = create(:question, questionnaire_id: @questionnaire_1.id, id: 2)
@question_1_3 = create(:question, questionnaire_id: @questionnaire_1.id, id: 3)
@question_1_4 = create(:question, questionnaire_id: @questionnaire_1.id, id: 4)
@question_1_5 = create(:question, questionnaire_id: @questionnaire_1.id, id: 5)
</pre>

==== Reviewers and Reviewees ====
* Objects involved 
[[participants]] 
[[teams]] 

* Code Implemented 
Reviewers (Participant):
<pre>
@reviewer_1 = create(:participant, can_review: 1)
@reviewer_2 = create(:participant, can_review: 1)
@reviewer_3 = create(:participant, can_review: 1)
</pre>

Reviewees (Teams):
<pre>
@reviewee_1 = create(:assignment_team, assignment: @assignment)
@reviewee_2 = create(:assignment_team, assignment: @assignment)
@reviewee_3 = create(:assignment_team, assignment: @assignment)
</pre>

==== Responses ====
* Objects involved 
[[response_maps]] 
[[responses]] 

* Essential Parameters to be configured 
reviewed_object_id = <target_assignment>.id ; 
reviewer_id = <target_reviewer>.id ; 
reviewee_id = <target_reviewee>.id ; 
Note: The response map is setup to determine the relationship between reviewer and reviewee of an assignment. 

* Code Implemented 
Response_maps:
<pre>
# response_map__<j> means reviewer #j's review to reviewee #i.
@response_map_1_1 = create(:review_response_map, reviewer_id: @reviewer_1.id, reviewee_id: @reviewee_1.id)
@response_map_1_2 = create(:review_response_map, reviewer_id: @reviewer_2.id, reviewee_id: @reviewee_1.id)
@response_map_1_3 = create(:review_response_map, reviewer_id: @reviewer_3.id, reviewee_id: @reviewee_1.id)

@response_map_2_1 = create(:review_response_map, reviewer_id: @reviewer_1.id, reviewee_id: @reviewee_2.id)
@response_map_2_2 = create(:review_response_map, reviewer_id: @reviewer_2.id, reviewee_id: @reviewee_2.id)
@response_map_2_3 = create(:review_response_map, reviewer_id: @reviewer_3.id, reviewee_id: @reviewee_2.id)

@response_map_3_1 = create(:review_response_map, reviewer_id: @reviewer_1.id, reviewee_id: @reviewee_3.id)
@response_map_3_2 = create(:review_response_map, reviewer_id: @reviewer_2.id, reviewee_id: @reviewee_3.id)
@response_map_3_3 = create(:review_response_map, reviewer_id: @reviewer_3.id, reviewee_id: @reviewee_3.id)
</pre>

Responses:
<pre>
# response__<j> means reviewer #j's review to reviewee #i.
@response_1_1 = create(:response, is_submitted: true, map_id: @response_map_1_1.id)
@response_1_2 = create(:response, is_submitted: true, map_id: @response_map_1_2.id)
@response_1_3 = create(:response, is_submitted: true, map_id: @response_map_1_3.id)

@response_2_1 = create(:response, is_submitted: true, map_id: @response_map_2_1.id)
@response_2_2 = create(:response, is_submitted: true, map_id: @response_map_2_2.id)
@response_2_3 = create(:response, is_submitted: true, map_id: @response_map_2_3.id)

@response_3_1 = create(:response, is_submitted: true, map_id: @response_map_3_1.id)
@response_3_2 = create(:response, is_submitted: true, map_id: @response_map_3_2.id)
@response_3_3 = create(:response, is_submitted: true, map_id: @response_map_3_3.id)
</pre>

=== Relevant Methods ===
==== db_query ====
This is the normal db query method, call this method will return peer review grades with given assignment id.
We will test this method in two aspect.
1. Test whether or not the grade return is right based on the specified algorithm.
2. We need to test the correctness of the query.

<pre>
context 'test db_query' do
it 'return average score' do
# reivewer_1's review for reviewee_1: [1, 2, 3, 4, 5]
create(:answer, question_id: @question_1_1.id, response_id: @response_1_1.id, answer: 1)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_1.id, answer: 2)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_1.id, answer: 3)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_1.id, answer: 4)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_1.id, answer: 5)
result = ReputationWebServiceController.new.db_query(1, 1, false)
expect(result).to eq([[2, 1, 60.0]])
end
end
</pre>

==== json_generator ====
This method will generate the hash format of the review, we will test this method by calling to and convert the result to json format the print to test its correctness.

<pre>
context 'test json_generator' do
it 'test 3 reviewer for one reviewee' do
# reivewer_1's review for reviewee_1: [5, 5, 5, 5, 5]
create(:answer, question_id: @question_1_1.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_1.id, answer: 5)

# reivewer_2's review for reviewee_1: [3, 3, 3, 3, 3]
create(:answer, question_id: @question_1_1.id, response_id: @response_1_2.id, answer: 3)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_2.id, answer: 3)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_2.id, answer: 3)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_2.id, answer: 3)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_2.id, answer: 3)

# reivewer_3's review for reviewee_1: [1, 1, 1, 1, 1]
create(:answer, question_id: @question_1_1.id, response_id: @response_1_3.id, answer: 1)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_3.id, answer: 1)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_3.id, answer: 1)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_3.id, answer: 1)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_3.id, answer: 1)

result = ReputationWebServiceController.new.json_generator(1, 0, 1)
expect(result).to eq({"submission1"=>{"stu2"=>100.0, "stu3"=>60.0, "stu4"=>20.0}})
#repeat for different answers
end

it 'test same reviewer for different reviewee' do
# reivewer_1's review for reviewee_1: [5, 5, 5, 5, 5]
create(:answer, question_id: @question_1_1.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_1.id, answer: 5)

# reivewer_1's review for reviewee_2: [3, 3, 3, 3, 3]
create(:answer, question_id: @question_1_1.id, response_id: @response_2_1.id, answer: 3)
create(:answer, question_id: @question_1_2.id, response_id: @response_2_1.id, answer: 3)
create(:answer, question_id: @question_1_3.id, response_id: @response_2_1.id, answer: 3)
create(:answer, question_id: @question_1_4.id, response_id: @response_2_1.id, answer: 3)
create(:answer, question_id: @question_1_5.id, response_id: @response_2_1.id, answer: 3)

result = ReputationWebServiceController.new.json_generator(1, 0, 1)
expect(result).to eq("submission1"=>{"stu2"=>100.0}, "submission2"=>{"stu2"=>60.0})
#repeat for different answers
end
end
</pre>

==== send_post_request ====
This method send a post request to peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms to calculate get the reputation result and use show the result in corresponding UI and update given reviewer's reputation. We will test this method based on the algorithm in the paper, first to test the result reputation value, second to test the update value in database.

Note that this method is not functioning due to the missing public & private key file for RSA encryption. The algorithm involving "expert grade" was also not implemented.
Thus, this method could not be properly tested. However we have given the template to create tests for future use.

<pre>
context 'test send_post_request' do
it 'failed because of no public key file' do
# reivewer_1's review for reviewee_1: [5, 5, 5, 5, 5]
# create 5 answers for 5 related questions
create(:answer, question_id: @question_1_1.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_2.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_3.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_4.id, response_id: @response_1_1.id, answer: 5)
create(:answer, question_id: @question_1_5.id, response_id: @response_1_1.id, answer: 5)

# choose hammer algorithm without expert grade(intructor's given grade)
params = {assignment_id: 1, round_num: 1, algorithm: 'hammer', checkbox: {expert_grade: "empty"}}
session = {user: build(:instructor, id: 1)}

expect(true).to eq(true)

# comment out because send_post_request method request public key file while this file is missing
# so at this time send_post_request is not functioning normally
# if it functions correctly, it will update the reviewer's reputation score according to the selected reputation algorithm.

# get :send_post_request, params, session
# expect(response).to redirect_to '/reputation_web_service/client'
end
end
</pre>

== Results ==
=== Testing Results ===

=== Coverage ===

== Future Tasks ==
As our team could not obtain the public/private key pair to access the reputation web service, we were only able get to the step prior to sending the JSON to the web service of reputation algorithms. Therefore, future steps are required to test on reputation system.

# Current setup is based on assignment_1, which uses lauw's algorithm to calculate reputation score. Future testing team need to setup similar object set for assignment_2, which uses hamer's algorithm.
# In send_post_request, there are references to specific assignments, such as 724, 735, and 756. They were put in to gather data for a paper published in 2015. They are no longer relevant and should be removed. (Need to replace by the comment Professor mentioned in the meeting today)
# Round 1 and round 2?
# The db_query violates the DRY principle as it repetitively calculates sum of the assignment. Such sum calculation should be handled in the assignment.rb.

== Collaborators ==
Jinku Cui (jcui23)

Henry Chen (hchen34)

Dong Li (dli35)

Zijun Lu (zlu5)

== References ==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://doi.org/10.1109/FIE.2015.7344292 Pluggable reputation systems for peer review: A web-service approach]

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:08:44Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.
[[File:Email.png|700px]]

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:08:16Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.
[[File:Email.png]]

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:07:35Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.
[[File:Email.jpg]]

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

File:Email.png

2021-11-07T02:06:57Z

Dli35: Dli35 uploaded a new version of File:Email.png

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:06:30Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:05:10Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

CSC/ECE 517 Fall 2021 - E2135 Email notification to reviewers and instructors

2021-11-07T02:04:14Z

Dli35:

This page provides a description of the Expertiza based OSS project.
__TOC__

==About Expertiza==

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project developed using the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. The application allows students to submit and peer-review learning objects (articles, code, websites, etc)[1]. Expertiza supports submission across various document types, including the URLs and wiki pages.

==Problem Statement==

<li> When an assignment or review approaches its deadline on Expertiza, students initially should receive deadline reminder emails at a specific time before the deadline that the instructor has preconfigured. Lack of this functionality sometimes results in students missing their assignment submission deadlines and thus losing marks. Students should receive this type of deadline reminder email. So this amendment to the project involves adding an asynchronous deadline reminder mailer to the application.

==Modified Files==
* app/models/due_date.rb
* test/models/due_date.rb
* db/migrate/20210319212323_create_delayed_jobs.rb

==Pre-config==

We use delayed jobs library to deal with the delayed email.
We need run following command in order to install '''delayed_job''' binary executable.

rails generate delayed_job

Then we need to run

RAILS_ENV=development bin/delayed_job start

This will start a backend server to deal with the delayed jobs.
If we want to stop, just run following.

RAILS_ENV=development bin/delayed_job stop

==Implementation approach==

'''1) Reminder email sent when assignment or review is approaching deadline:'''
In the '''due_date.rb''' file, whenever a new due date is created or an existing due date is updated, the '''start_reminder''' method will be fired which will eventually be added to the delayed_job queue. This job will be executed at a preconfigured time before the deadline, where it will fire the method '''reminder''' which will be added to the delayed job queue by the handle_asynchronously method of gem '''delayed_job_active_record'''. Inside the reminder method, we will fetch three attributes - '''''assignment_id''''', '''''deadline_type''''', '''''due_at'''''. These three attributes will be used to decide the deadline type ( submission or review or teammate review ), fetch the participant email for that assignment, fetch the deadline threshold and at the end send the email reminder at a specified threshold time before the deadline which will contain all the details such as assignment names link to assignment and assignment type ( submission or review or teammate review).

'''2) Implement code:'''

def create_mailer_object
Mailer.new
end
def create_mailworker_object
MailWorker.new(self.parent_id, self.deadline_type, self.due_at)
end

# main function to start email reminder
def start_reminder
puts when_to_run_reminder
if self.changed?
@extra_param = self.parent_id.to_s + "," + self.deadline_type_id.to_s
# first deleted existed delayed jobs with same parent_id(which is assignment id actually)
Delayed::Job.where(extra_param: @extra_param).each do |job|
job.delete
end
# add a delayed job to the delayed job queue, the job will run at what when_to_run_reminder return
run_at_time = when_to_run_reminder
if run_at_time >= 0.seconds.from_now
self.delay(run_at: run_at_time, :extra_param => @extra_param).reminder
end
end
end
####
def reminder
deadline_text = self.deadline_type if %w[submission review].include? self.deadline_type
deadline_text = "Team Review" if self.deadline_type == 'metareview'
mail_worker = create_mailworker_object
email_reminder(mail_worker.find_participant_emails, deadline_text) unless mail_worker.find_participant_emails.empty?
end
####
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.parent_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
emails.each do |mail|
Rails.logger.info mail
end
Mailer.delayed_message(bcc: emails, subject: subject, body: body).deliver_now
end

# after duedate - threshold hours, then we can send the reminder email
def when_to_run_reminder
hours_before_deadline = self.threshold.hours
result = (self.due_at.in_time_zone - hours_before_deadline).to_dateti

==Automated Testing using RSPEC==

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the mail is enqueued upon the firing of the reminder method.

We have used Rspec for testing the delayed_job functionalities. Using the test-driven development (TDD) approach, we have added a Rspec test which checks whether the call for reminder method is enqueued upon the firing of the '''start_reminder''' method, with the proper scheduled execution time.

'''NOTE''': All the reminder mails except the ones for the reviewer are sent to '''''expertiza.development@gmail.com''''' ,as this is already set in the development environment.

==Steps to verify Functionality==

*Test Email
Email: expertiza_test123@outlook.com
Password: password98@

*Test Expertiza account
name/login: Email_Test_ID1
password: password98@

1. Change the existing assignment's due date
First login in as instructor6, and go to Manage → Assignments

[[File:manage_page.png|700px]]

Then click change edit in the right side menu under **Actions**.
Go to Due dates and change the **reminder** and **Due & Time**, and click save.

[[File:edit_assignment.png|700px]]

Then you should receive email at expertiza_test123@outlook.com in the appropriate time. For this example,16 hours before 2021/10/25 04:08 (US Eastern time zone), the system shall send a reminder email to expertiza_test123@outlook.com.

2. Create a new assignment with valid due date
Create a new Assignment

[[File:create_assignment.png|700px]]

Assign a valid due date and reminder hours. Valid means that the due data minus reminder hour should be a time that is later than the current time. For example, if you set due date to 10:00pm on 10/25/2021 and set reminder hours to 8 hours, then you won't get the email if current date time has passed 2:00pm on 10/25/2021.

[[File:new_date.png|700px]]

Then add the previous user login to it as a participant

[[File:add_participate.png|700px]]

Because the participants was added later, so we need to change the due date once more in order to trigger the email notification jobs.

[[File:Example.jpg]]

==Additional Links==

*Git forked repository link: https://github.com/CuiJinku/expertiza/tree/beta
*VCL deployment: http://152.7.176.117:8080/
*Pull request link: https://github.com/expertiza/expertiza/pull/2099

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/CuiJinku/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://relishapp.com/rspec Rspec Documentation]

==Team==
[mailto:dli35@ncsu.edu Dong Li] 
[mailto:ldu2@ncsu.edu Liwen Du] 
[mailto:jcui23@ncsu.edu Jinku Cui]

CSC/ECE 517 Fall 2021 - E2168. Testing - Reputations

2021-11-04T00:18:38Z

Dli35: /* ReputationController_spec */

== Project Overview ==

=== Objectives ===

Our objectives for this project are the following:
* Double and stub an assignment, a few submissions to the assignment, under different review rubrics
* Manually calculate reputation scores based on paper "Pluggable reputation systems for peer review: A web-service approach"
* Validate correct reputation scores based on different review rubrics generated by reputation management VS manual computation of reputation score expectation

=== Files Involved ===
*reputation_web_service_controller_spec
*hamer_spec
*<del>lauw_spec (?)</del>

== Test Plan ==

=== Testing Objects ===
Following objects will be created and confined for the purpose of testing.

*[[assignments]] 
*[[submission_records]] 
*[[questionnaires]] 
*[[assignment_questionnaires]] 
*[[participants]] 
*[[teams]] 
*[[teams_users]] 
*[[users]] 
*[[responses]] 
*[[response_maps]]

=== Relevant Methods ===
==== hamer_spec ====
*calculate_weighted_scores_and_reputation
This is the main method of Hamer algorithm. Reviewers inaccuracy will be calculated and updated from past average, then weight of the review will be adjusted based on the inaccuracy. We plan to create two contexts to test it as below: 1. When the review is the same from average, reviewer’s inaccuracy should be minimized and the review’s weight should be maximized; 2. When the review is furthest from average, reviewer’s inaccuracy should be maximized and the review’s weight should be minimized.
*converged?
This method ensures the final result of calculated reviews' weights is converged, as the algorithm requires. A simple fixture test can be done by giving the method a set of input data and compare it to expected result.
==== ReputationController_spec ====
*db_query
This is the normal db query method, call this method will return peer review grades with given assignment id.
We will test this method in two aspect. 1. Test whether or not the grade return is right based on the specified algorithm.
2. We need to test the correctness of the query.

*db_query_with_quiz_score
This is the special db query, call this method will return quiz scores with given assignment id. We will test this method with same logic as the first one.

*json_generator
This method will generate the hash format of the review, we will test this method by calling to and convert the result to json format the print to test its correctness.

*client
client method will fill many instance variables with corresponding class variables, we need to test this method with send_post_request.

*send_post_request
This method send a post request to peerlogic.csc.ncsu.edu/reputation/calculations/reputation_algorithms to calculate get the reputation result and use show the result
in corresponding UI and update given reviewer's reputation. We will test this method based on the algorithm in the paper, first to test the result reputation value, second to test the
update value in database.

== Results ==
=== Collaborators ===
Jinku Cui (jcui23)

Henry Chen (hchen34)

Dong Li (dli35)

Zijun Lu (zlu5)

==References==
#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[https://doi.org/10.1109/FIE.2015.7344292 Pluggable reputation systems for peer review: A web-service approach]