CSC/ECE 517 Fall 2019 - E1941. Issues related to topic deadlines

This wiki page is for the description of changes made under E1941- Issues Related to Topic Deadlines OSS assignment for Fall 2019, CSC/ECE 517.

About Expertiza

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.

Issues in the Current Project

There is no deadline that specifies when a team can drop their topics. Due to this, few problems occur: The team might drop a topic later on which is close to the submission date and this results in topic not being assigned to any other team on time such that they can submit their assignment. Also, if one team was wait listed for the same topic which the other team dropped closer to the submission deadline, then the first wait-listed team will be assigned to the dropped topic which is not desirable as they might be working on their assigned topic for long time.

There are different topics on which students can work during different times. This type of assignment is known as Staggered- deadline assignment in which different topics have different submission and review deadlines. For these assignments too, there is a need for "Drop Topics Deadline". In the current implementation, the drop topic deadline is same for all the topics in a Staggered Deadline assignment which is not desirable.

The following points were identified in the problem statement, regarding when a team should be dropped automatically:

- They get a topic they were on the waitlist for

- The last member leaves the team

- A drop-topic deadline passes, or a submission passes

Out the three requirements, first two features are already implemented. We have implemented the last one, details are given below.

Current Implementation

Functionality

The instructor cannot add a Drop Topic Deadline for topics

The instructor has the functionality to specify Review and Submission deadlines, while editing a topic. This page however, doesn’t have any provision on the form to let the instructor add Drop Topic Deadlines. According to the current implementation, s/he can add in a date for Drop Topic Deadline as well.

The wait-list for a topic doesn’t get cleared on reaching the Drop Topic Deadline

Whenever the Drop Topic Deadline is crossed, we expect:

- The waitlist queue for the topic is cleared

- The teams are not permitted to drop the assigned topics anymore.

Previously, we had no job scheduled for dropping the waitlists. This segment involves a queue set-up for job scheduling (Sidekiq), which is not fully functional in the current implementation. To fix the issue, we have written the code for dropping teams from the waitlist which will work as expected once the queue starts working.

Calculation of Drop Topic Deadline

Issues could occur if user accidently sets a drop topic deadline which is after the submission date of the topic. This case can be handled by checking the submission deadline of the topic against the date user sets. The flow works as follows:

- Case: 'no deadline date is set on topic' - the topic submission deadline is used to clear the teams waitlisted for the topic (but the drop topic deadline is not set)

- Case: 'date prior to the topic submission deadline is set as drop topic deadline' - the date set by user is set and used for clearing waitlisted teams

- Case: 'date after the topic submission deadline is set as drop topic deadline' - the date set as submission date is logically correct date in this case and gets set as the drop topic deadline as well and is used to clear the teams wailisted

Drawbacks and Solutions

Problem 1: No provision for taking user input for Drop Topic Deadline and persisting it in the database. Although the current schema supports a Topic/Assignment having drop Topic Deadlines, there is not form input field for the same.

Solution: The implementation has been changed in such a way that the user can input the date, and the same is persisted under the due_dates table.

Below is the code snippet from app/models/topic_due_date.rb, dealing with the above change:

  # adds a new deadline if not present,
  # updates the date if already present
  def self.upsert_drop_deadline_job(assignment_id, topic, drop_topic_input)
    deadline_type_id = DeadlineHelper::DEADLINE_TYPE_DROP_TOPIC
    drop_topic_date = TopicDueDate.where(parent_id: topic.id, deadline_type_id: deadline_type_id).first rescue nil

    calc_drop_topic_date = get_drop_topic_deadline_date(assignment_id, topic.id, drop_topic_input)

    # if drop topic deadline is not in db, make an entry
    if drop_topic_date.nil?
      # if user sets a date before creating the first entry
      due_at = (drop_topic_input.nil? || drop_topic_input.blank?) ? nil : calc_drop_topic_date

      # add delayed job to drop waitlisted teams after deadline passes
      delayed_job_id = modify_delayed_job(topic.id, calc_drop_topic_date, nil, false)

      TopicDueDate.create(due_at: due_at,
                          parent_id: topic.id,
                          deadline_type_id: deadline_type_id,
                          type: DeadlineHelper::TOPIC_DEADLINE_TYPE,
                          delayed_job_id: delayed_job_id)
    else
      update_delayed_job = true

      if !drop_topic_date.due_at.nil? && (drop_topic_input.nil? || drop_topic_input.blank?)
        # if drop topic deadline is deleted
        due_at = nil
      elsif drop_topic_date.due_at.nil? && !(drop_topic_input.nil? || drop_topic_input.blank?)
        # if drop topic deadline is entered first time
        due_at = calc_drop_topic_date
      elsif !drop_topic_date.due_at.nil? && 
          drop_topic_date.due_at.to_datetime.strftime(DeadlineHelper::DATE_FORMATTER_DROP_DEADLINE) != calc_drop_topic_date.strftime(DeadlineHelper::DATE_FORMATTER_DROP_DEADLINE)
        # if drop topic deadline is updated
        due_at = calc_drop_topic_date
      else
        # if updated date is same as existing, don't update the delayed job
        update_delayed_job = false
      end

      if update_delayed_job
        delayed_job_id = modify_delayed_job(topic.id, calc_drop_topic_date, drop_topic_date.delayed_job_id, true)
        drop_topic_date.update_attributes(due_at: due_at, delayed_job_id: delayed_job_id)
      end
    end
  end

  # check the user input date and return the accurate deadline for dropping topics
  private
  def self.get_drop_topic_deadline_date(assignment_id, topic_id, drop_topic_input)
    expected_drop_deadline_date = DueDate.get_deadline_to_drop_topic(assignment_id, topic_id)

    if drop_topic_input.nil? || drop_topic_input.blank?
      return expected_drop_deadline_date
    end

    drop_topic_date = DateTime.parse(drop_topic_input)

    # if user sets drop topic deadline ahead of submission date, use submission date
    return (drop_topic_date.utc > expected_drop_deadline_date.utc) ? expected_drop_deadline_date : drop_topic_date
  end

Problem 2: Waitlisted topics not getting dropped after the Drop Topic Deadline. For executing the timed jobs, Expertiza uses Sidekiq, and creates jobs for every Topic. Referred to as delayed_jobs, it currently has some defined for tasks like dropping a team when the last member leaves, etc.

Solution: We need a job to be executed on the day of the specified Drop Deadline, which would drop the waitlist. To resolve this, we added another job which gets created on every new Topic creation, to be executed at the Drop Topic Deadline. In case there is an update on a previously defined deadline by the instructor, we fetch the existing job, using the delayed_job_id, and modify the same as per the new Deadline.

Edge cases addressed:

1. In case the Topic Drop Deadline is not defined, we set the default based on the following dates, whichever is earlier/populated:

a. Topic Submission Deadline

b. Assignment Drop Topic Deadline

c. Assignment Submission Deadline

2. In case of an updation on Drop Deadline, we check if the new entry is different from the existing date, and reschedule the job only if it is, thereby avoiding any redundant updates.

3. Whenever the user-entered Topic Drop Deadline is earlier than the Submission Deadline, we set the date as per (1).

Below is the code snippet from app/models/topic_due_date.rb, dealing with the above change:

  # this method either adds a new job to the queue or deletes
  # an existing job and replaces it with a new one
  private
  def self.modify_delayed_job(topic_id, drop_topic_date, delayed_job_id, job_present)
    if job_present
      remove_job_from_queue(delayed_job_id)
    end

    mins_left = calculate_mins_left(drop_topic_date)
    return add_job_to_queue(mins_left, topic_id, "drop_topic", drop_topic_date)
  end

  private
  def self.calculate_mins_left(drop_topic_date)
    curr_time = DateTime.now
    ((curr_time - drop_topic_date) * 24 * 60).to_i
  end

  private
  def self.add_job_to_queue(min_left, topic_id, deadline_type, due_at)
    delayed_job_id = MailWorker.perform_in(min_left * 60, topic_id, deadline_type, due_at)
    return delayed_job_id
  end

  private
  def self.remove_job_from_queue(job_id)
    queue = Sidekiq::ScheduledSet.new
    queue.each do |job|
      current_job_id = job.args.first
      job.delete if job_id == current_job_id
    end
  end

Files modified in Current Project

1. app/helpers/sign_up_sheet_helper.rb

2. app/models/due_date.rb

3. app/models/signed_up_team.rb

4. app/models/waitlist.rb

5. app/controllers/sign_up_sheet_controller.rb

6. app/helpers/deadline_helper.rb

7. app/mailers/mail_worker.rb

8. app/models/topic_due_date.rb

9. spec/controllers/sign_up_sheet_controller_spec.rb

Test Plan

RSpec:

It was discussed that project does not require any Unit Test cases to be implemented as the changes are mostly visible in views.

Of all the code changes, we have one controller modification that had a rspec test case. We modified the same as per the changes we made to the code.

Further, all the additional methods created are private methods and so, we've not created any rspec test cases for those.

UI Testing:

[Problem 1]

1. Login to Expertiza using username: instructor6 & password: password.

2. Select an existing course/Create a course and an assignment under that course. Check 'Staggered deadline assignment?' for the assignment.

3. Create topics under the assignment.

4. Go to Assignment -> Topics. Click 'Show start/due date' at the bottom of the screen.

5. Enter 'Drop Topic Deadline' and hit 'Save start/due date'.

6. Reopen the assignment topic to cross verify if the deadline has been saved.

Caveat: While creating an assignment the page would error out, but the assignment does get created, which is not related to any of the changes we have done. In case unable to do so, use 'MadeUp Problem' with Creation Date of 'Aug 30, 2017 - 9:03 AM'

[Problem 2]

When a deadline is assigned for a topic, we create a job (identified with delayed_job_id) to be executed on the deadline date. The module for job and queue set-up on expertiza is not fully functional, and hence this cannot be tested directly. However, we have written the code for job scheduling, here's a snapshot of the due_dates table with the delayed_job_id populated: