CSC/ECE 517 Fall 2014/OSS E1451 las

E1451 - Create Mailers for All Email Messages

Overview

Expertiza is an Open Source Rails application designed to support team/individual student projects which involves team creation, project sign up, project submissions, document upload (almost any format), and carry out multiple peer review processes for learning and educational purpose. The rationale behind Expertiza is that the students should not only be consumers but also creator of educational resources. Based on this philosophy, the tool is extensively managed and consumed by students of class CSC517 Object Oriented Programming Languages and Systems course at NC State University as their main project management tool. This Open Source application can be cloned from Github, the latest active branch is "Rails 4".

Action Mailer supports email functionality within a Rails application, which involves sending and receiving emails from and to an application. The scope of the Action Mailer within Expertiza is limited to sending emails only which we have proposed to extend to 'Receiving' under the heading ‘Future Work’.

Project Requirements

Problem Definition

Design strategy has been one of the major issue for erratic behavior of Mailer within Expertiza. From past few years the Expertiza has been combined with various enhancements and the code has been changed by different teams, the difference in the implementation practices has introduced major design flaws within some components (including Mailer) of the application. Another major issue is the migration issue from Rails 3 to Rails 4 as a result of which many of the unsupported features for Rails 4 have stopped working.

Requirements

The aim of this Mailer project is to bring consistency in the design implementation and restore the e-mailer and e-mail messages capability from Projects E701 and E729, and bring back the new features created by Project E916. High-level requirements are listed below:

1. Restore features and functions in Expertiza that trigger email messages

2. Create mailers for all email messages for both asynchronous events and synchronous events

3. All emails should be sent via ActionMailer

4. Emails should have different class

Significant amount of time has been spent on correcting bugs and errors in the program before the email features can be restored, and therefore this wiki also documents the changes our team has made and provides a guideline for future development and enhancement.

Mailer in Expertiza

Type of Mailers

Expertiza has two different Mailer implementations based on type of events: Synchronous events and Asynchronous events.<ref> Synchronous and asynchronous events</ref> The synchronous events refer to events that should be immediately notified. For example, email should be sent immediately after a review; or after a feedback of the review has been posted; or a user/team is assigned a project, or a social bookmark have been added, etc. Asynchronous Events are a series of timed events which gets triggered at some fixed times, such as weekly quiz reminders, deadline approaching reminders or in fact Sign Up topic availability reminder is a very explanatory example of Asynchronous Events.

Below mentioned are the changes made in each of the Mailer implementation and in related files.

Synchronous Mailer and Issues

All synchronous events are triggered by controllers and the call is made to app/mailers/mailer.rb (formally as model in /app/models/) via the sync_message() method call. Here mailer.rb extends ActionMailer class. Because part of the email implementation was done on Rails 3 platform, to make it compatible for Rails 4, revision has been made. The required email template has been added for both generic_message and sync_message method calls. Here is an example of synchronous event email that simulates an event when the work you reviewed before has been revised.

1. app/models/mailer.rb has been moved to app/mailers/mailer.rb

2. Two view templates have been added to app/views/mailer directory. They are sync_message.html.erb and sync_message.text.erb

    <!DOCTYPE html>
    <html>
    <head>
      <meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
    </head>
    <body>
    <%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>

    <hr>

    This message has been generated by <A HREF="http://expertiza.ncsu.edu">Expertiza</A><BR/>
    http://expertiza.ncsu.edu

    </body>
    </html>

3. Correct DOCTYPE and html codes have been added to app/views/mailer/generic_message.html.erb

4. The following line in the sync_message(defn) method has been commented out from app/mailers/mailer.rb in order to encode email correctly.<ref>Read MIME Wiki page for email content type system</ref>

   content_type: "text/html"

5. The following method calls in the following files have been changed.

a) app/controllers/grades_controller.rb
b) app/models/courses_users.rb
c) app/models/participant.rb

Original:

   Mailer.deliver_message(

Changed:

   Mailer.sync_message(

Asynchronous Mailer and Issues

The asynchronous events are registered to Delayed::Job object and a daemon-rails server must be activated to carry out the task. All events are called to DelayedMailer class in /app/emailers/delayed_mailer.rb via one of the following methods: email_reminder(), mail_assignment_participants(), mail_reviewers(), mail_metareivewers(), getTeamMembersMail(), and mail_signed_up_users(). The DelayedMailer class acts as a wrapper class (i.e., adapter pattern<ref>The Adapter Pattern (Design Pattern) page</ref>) to the Mailer class in the /app/mailers/mailer.rb via the delayed_message() method call. These events are added to the queue by Delayed::Job.enqueue() command. Each event as record is stored in the delayed_job table, which can be browsed on RubyMine. Because all of the asynchronous events are controlled by the assignment controller, events are triggered by the add_to_delayed_queue() method. Each time when an assignment is updated or saved, all of the event queues associated with such assignment are refreshed. We found this is not very efficient but it has worked as expected.

Other Issues Addressed

Data migration issue

The database migration issue occurs when carrying out "rake db:migrate" command. We found the mirrored image from the existing database still has incorrect table fields. The problem may not appear until later during feature testing. Therefore, our team decided to fix the data migration issues from ground up. In total, more than 36 files have been fixed. Most of them are database migration files. To eliminate future table creation issues with mysql2 gem, we have also changed the integer byte length of MySQL tables from 10 to 8 and have revised the SQL query methods inside the migration files to improve efficiency. The methods missing from the app/models directory related to site_controller have been restored for controller_action, menu_item, permission, role, and site_controller models.

   def self.find_or_create_by_name (params)
     MenuItem.find_or_create_by(name: params)
   end

Changes to data migration files are illustrated as follows:

Original:

   :integer, :limit => 10, :default => 0, :null => false

Changed:

   :integer, :limit => 8, :default => 0, :null => false

Detailed code changed can be found at Github.

Assignment Controller

Each due date for submission, review, meta review, sign up, and drop topic were missing under the Assignment edit page due to the bugs in its views and partials files. The JavaScript implementation was done via JQuery gem <ref>Wiki page on JQuery</ref> but it has not been parsed correctly. After the issue has been fixed, now the correct due dates are displayed correctly.

The following changes have been made:

1. All *.rhtml files in app/views/bookmarks have been renamed to *.html.erb for compatibly reason.

2. The following lines have been added to Gemfile

   gem 'jquery-datetimepicker-rails'
   gem 'graphr'
   gem 'daemons'

3. The following parsing errors in app/controllers/assignment_controller.rb have been corrected.

Original:

   duatedates[i].due_date(:db).to_s

Changed:

   duatedates[i].due_date.to_s(:db)

4. The following code in ap/views/assignments/edit/_rubrics.html.erb has been changed.

Original:

   <%= questionnaire(@assignment, 'ReviewQuestionnaire').to_json.html_safe %>.review_questionnaire
   <%= assignment_questionnaire(@assignment, 'ReviewQuestionnaire').to_json.html_safe %>.assignment_questionnaire

Changed:

   <%= questionnaire(@assignment, 'ReviewQuestionnaire').to_json.html_safe %>
   <%= assignment_questionnaire(@assignment, 'ReviewQuestionnaire').to_json.html_safe %>

Testing

RSpec test cases

RSpec is a behavior-driven development (BDD) framework for Ruby programming language. Inspired by JBehave, it resembles a natural language specification. To test the emailer features, we have restored RSpec test cases for both delayed_mailer.rb (asynchronous maier) and mailer.rb (synchronous mailer). Because the existing RSpec files were developed in RSpec 2.x, upgrade is required. Ideally, the files in /test/fixtures should be used to populate the test data for RSpec, and therefore we did not restore all the files in /spec/fixtures/ directory. Currently, the fixture<ref>Read Wiki Unit Testing page for more details</ref> files are not completed and loading them into the test database may cause failure for the test, so we recommend to comment out the following line in the /spec/rails_helper.rb

   #config.global_fixtures = :all

Future developers must take extreme care to make sure all fixture files work before enable this option.

1. The two test files listed below have been added spec/ directory:

a) spec/integration/delayed_mailer_spec.rb
b) spec/models/mailer_spec.rb

The following code has been added to each test case:

   Delayed::Job.delete_all

2. The following code has been added to spec/rails_helper.rb in order for the RSpec to pass the test.

   ENV["RAILS_ENV"] ||= 'test'
   require File.expand_path("../../config/environment", __FILE__)
   require 'rspec/rails'
   require 'rspec/autorun'
   require 'capybara/rspec'
   require 'capybara/rails'

Installation for Reviewers

Because asynchronous events require a daemon running in the background and the new Delayed::Job class for Rails 4 has additional fields, the following steps are recommended.

1. Install Expertiza and execute 'bundle install'

2. Create database and perform data migration using 'rake db:migrate'

3. Execute 'rails generate delayed_job:active_record' at the prompt. This will create a delalyed_job file in /bin directory.

4. Execute 'rails generate delayed:upgrade' if delayed_job has been created before.

5. Enter your smtp server account settings in config/environments/production.rb file. You may find an example in config/environments/developments.rb file.

6. Start the rails server by entering

   rails server

5. Start the delayed_job server by executing

   RAILS_ENV=production bin/delayed_job start

Known Issues and Proposed Future Work

1. The due date for signup activity may not display correctly. To refresh, users must click the "review round" button or the "save" button again.

2. Each time an assignment is saved all delayed_job tasks will be deleted and refreshed even if the dates have not been changed. This may not be very efficiency and may cause problems in the future. Checking mechanism should be built in to verify if any date has been changed.

3. DateTimePicker field could be accidentally changed by mouse click.

4. Prioritization in the Delayed Job function in case of multiple email load

5. Extending functionality to include Weekly/Monthly digest that can consist of users WebAssign summary.

6. In-built messaging (Inbox, Sent, Draft etc.) functionality in addition to email for easy tracking of outgoing and incoming messages.

7. Extending Expertiza's scope to include 'Receive' email functionality.

References

<references/>

Expertiza + ActionMailer<ref>http://wiki.expertiza.ncsu.edu/images/d/d4/Logo.jpg</ref>
Name	ActionMailer + Expertiza
Category	Open Source Software