CSC/ECE 517 Fall 2016 E1708: Improvements to staggered-deadline assignments

From Expertiza_Wiki
Jump to navigation Jump to search

Introduction

Expertiza is a web based application which can be used by instructors to assign tasks to students. Expertiza also allows students to form teams among each other, perform peer reviews on assignments and projects. It gives flexibility to instructor to allow students to see other student's work with intended limitations and impose required restrictions on student access. Expertiza has been developed as an Open Source Software (OSS) on Ruby on Rails framework.

Motivation

Expertiza open source development program has fetched many contributions from faculties and students of different universities and thus, it incorporates a wide variety of Ruby coding styles, and this has been a reason why Expertiza project is kept as CSC/ECE 517 final project. The students have been given guidelines on Ruby coding styles and many problem statement includes tasks like refactoring and writing tests which can improve uniformity in coding style. The opportunity to work on an open source project like Expertiza is also expected to provide student an introductory experience of understanding larger programs and making required contributions to them. The writing assignment along with programming assignment is expected to provide project documentation experience to students.

Description of Project

The project numbered E1708 and titled ‘Improvements to Staggered-Deadline Assignments’ is intended to make contributors or students understand the implementation of assignments and due dates models and tables, so that they can improve on the methodology which is used to assign the due dates to assignments and topics. A staggered-deadline assignment is an assignment in which different topics have different deadlines. It can be pretty useful if different topics within an assignment can have different deadlines for submission as well as reviews. The project has been divided into three problem statements and each of them is addressing an individual functional requirement related to the staggered deadline. The statements and the proposed approaches to meet the requirements are explained below as implementation.

Project Requirements

The following issues can arise with a staggered-deadline assignment.

Problem 1: Staggered-deadline topic available for selection past its deadline

For a topic with multiple slots that is posted, if it is not selected in the first round, it is still available for selection in a later round. The team that selects the topic then is unable to submit their work as it is past deadline. A solution to this is to prohibit anyone from signing up for a topic whose submission deadline has passed.

Problem 2: Manual submission and review deadline entry required for all topics

Currently system requires instructor to enter submission and review deadlines for all topics. A probable solution to this is to allow the instructor to apply one deadline entry to a set of topics and also let the system calculate subsequent dates for subsequent project phases like review and re-submission, based on an initial deadline entry.

Problem 3: New submissions not identifiable among pool of already graded entries

Currently there is no way to identify new submissions among list of entries for which grading has already been done. There should some marker or way to distinguish new entries.

Analysis and Implementation

For Problem 1, the solution that we have chosen to implement is fix 1. We will prevent anyone from signing up for a topic whose submission deadline has passed. Currently this is happening, but only in case of assignments which do not have staggered deadlines. This logic can be found in the ‘list’ method in the sign_up_sheet_controller.rb class. If the assignment is not a staggered deadline assignment and if its submission deadline has passed, then we prevent students from signing up for any topic in that assignment. Bellow code snippet indicates the same. 

unless assignment.due_dates.find_by_deadline_type_id(1).nil?
   if !assignment.staggered_deadline? and assignment.due_dates.find_by_deadline_type_id(1).due_at < Time.now
   @show_actions = false

We will have to replicate a similar logic for staggered deadline assignments. The problem here is that for staggered deadline assignments, different topics might have different deadlines. So, we will have to check deadlines for topics on an individual basis. Deadlines for individual topics are stored in a table called due_dates with the parent_id as the identifier of the particular topic in place of the assignment identifier. A logic for getting the due dates of individual topics for staggered deadline assignments is found in the method check_topic_due_date_value in the module SignUpSheetHelper. 

topic_due_date = TopicDueDate.where(parent_id: topic_id, deadline_type_id: deadline_type_id, round:review_round).first rescue nil
if !topic_due_date.nil?
   due_date = topic_due_date.due_at
else
   due_date = assignment_due_dates[review_round - 1].due_at.to_s

First we check if the topic has a specific different deadline set by the instructor, and if not, then its deadline will be the same as the common deadline for the assignment. This way, we will filter out topics whose submission due dates have passed and for each of them, we will disable the selection option (Green color tick mark).

Implementation: As mentioned, we are going to have to check deadlines on a topic-by-topic basis and so in the \app\views\sign_up_sheet\_all_actions.html.erb file, where we actually decide whether a student can sign up for a particular topic, we have implemented the following check to ensure that the submission deadline for that topic has not yet passed. We do this only for staggered deadline assignments as for the other assignments, this check has already taken place. 

!@assignment.staggered_deadline? || Time.now < get_topic_deadline([@assignment.due_dates.find_by_deadline_type_id(1)],topic.id,1,1)

Design Principles used:

  1. Code re-usability: In order to ensure that we did not duplicate code, we used existing code in the check_topic_due_date_value method with some slight refactoring so that it could return us the dates in the required form.

Bellow is the a flow diagram indicating how deadlines are chosen for various topics:

In Problem 2, the current implementation of the system requires the instructor to manually enter deadlines for the three submission review rounds which can be a painstaking task. Our requirement is to provide the instructor with the option to select a relative date for the submission review deadlines based on the Round1 submission deadline. The instructor would however like the ability to manually enter a custom date according to his needs.

One way of dealing with the above problem would be to provide an option for the other date fields for a given assignment to chose a relative date based on the initial submission date in addition to the ability to override and manually enter a custom date. Example: 3 days after Submission deadline, 6 days after re-submission deadline. This approach of implementation will only require changes to the _due_dates.html.erb view for the assignments controller.

Changes will have to be made to the due_dates_table in the _due_dates.html.erb file. Changes have to be made to the datetime_picker_roundtype_roundnumber id. We could provide the user with a drop down menu listing a possible set of relative dates to chose from, in addition to the ability to manually enter a custom date. This could be done on the view side by modifying the date fields and adding the necessary JavaScript so that appropriate date responses are filled into the corresponding date fields when an option is chose from the dropdown menu.

For Problem 3, the grader needs to know about the new reviews done by the author since the last time grading was done. Grades for reviews are stored in the grade_for_reviewer column in the participant table. Reviews done by a Reviewer for a particular reviewee are mapped in the response_maps table and the actual data regarding the reviews (round no, comments, date, etc.) are stored in the responses table with the map_id as the foreign key. In order to implement our solution, we will add a new review_last_graded_date column to the participant table to store when the grading was last done. If the grader changes the grade, then the review_last_graded_date will change to reflect the latest date.

Now, when the grader views the review_report for a particular participant,

  1. The code gets the review_last_graded_date for this participant
  2. Then it checks the responses table for the reviews that this participant has done for that assignment and compares the updated_at date for each of those reviews with the review_last_graded_date.
  3. If the updated_at date for any of those reviews is greater than the review_last_graded_date, we can change the color of the link which is displayed for that review to blue color to indicate that the participant has entered this new review, and that it has not yet been graded.

Implementation

  1. In the method 'save_grade_and_comment_for_reviewer' , in the review_mapping_controller, we save the grading date along with the actual grade and comments
  2. In the view app\views\review_mapping\_review_report.html.erb, we add a new method 'get_team_color' which actually performs the steps displayed in the flow diagram. It does this for each review that a particular participant has done.


Use Cases

Problem 1

For both the parts in this problem, we have chosen CSC/ECE 506, Spring 2015 as the course and Chapter 11-12 madeup exercises as the assignment. The first step will be to make this assignment a staggered deadline assignment. For doing this edit the assignment and under the General tab, make this a staggered deadline assignment.

Part1: All submission due dates are in the past, so no topic is available for signup

  1. Login as instructor6
  2. Impersonate student5695
  3. In the Assignments section, go to assignment Chapter 11-12 made up exercises
  4. Click on Sign-up sheet
  5. No topic should be displayed for signup

Part2: Change due date to after today for a topic, now it should be available for signup

  1. Login as instructor6
  2. Go to Manage - -> Courses and select CSC/ECE 506, Spring 2015
  3. Edit the assignment Chapter 11-12 madeup exercises
  4. Go to Topics tab and scroll down to the bottom and click on Show start/due date
  5. Make the submission deadline for topic 11.2 as any date after today
  6. Now Impersonate the student5695 and go to his sign_up sheet
  7. This topic 11.2 should be available for signup. All others remain unavailable

Problem 2

Part1: Instructor wants to assign default deadlines to a staggered assignment

In this case, the default deadlines for the assignment will be filled automatically based on the submission date for Round 1. In this case no further input from the instructor is required apart from selecting the round 1 submission deadline

  1. Login as instructor6
  2. Go to assignments and either create a new one or edit a pre-existing one.
  3. Change assignment type to staggered.
  4. Chose a submission deadline for round 1.
  5. Deadlines for all other rounds will be autopopulated.

Part2: Instructor wants to enter custom deadlines for a staggered assignment

In this case, the instructor has two options. He can either chose a deadline from a list of other relative deadlines from the dropdown menu for a particular round or all rounds, or he can decide to override the system enter a date manually by selecting the “Custom Date” option.

  1. Login as instructor6
  2. Go to assignments and either create a new one or edit a pre-existing one.
  3. Change assignment type to staggered.
  4. Chose a submission deadline for round 1.
  5. Deadlines for all other rounds will be autopopulated.
  6. Change all the deadlines for the other rounds according to your requirements by choosing the "Custom Date" option

Part3: Instructor wants to enter a custom deadline only for a particular round

In this case, the instructor can set the submission deadline for round 1. This would result in all deadlines being populated relatively according to the pre-set values. The instructor can then, if need be, modify the deadline for a particular round by either selecting a different relative date or by entering a custom date manually.

  1. Login as instructor6
  2. Go to assignments and either create a new one or edit a pre-existing one.
  3. Change assignment type to staggered.
  4. Chose a submission deadline for round 1.
  5. Deadlines for all other rounds will be autopopulated.
  6. Change all the deadlines for the necessary round by either entering a manual date or by choosing a different relative date from the dropdown.

Problem 3

Part 1: To check if the color changes to blue when a new review is entered after the grading has been done.

  1. Login as instructor6
  2. Go to courses tab
  3. For CSC/ECE 506, Spring 2014, go to edit assignments and choose New problems B
  4. Edit this and make it a staggered deadline assignment. Also change the review due dates for the first round to any date beyond today
  5. Now go back to assignments page and go to view review report for this assignment
  6. We focus on student 5059 for this case.
  7. This student has one review, go ahead and grade this review.
  8. Now impersonate student5059
  9. Go to the assignment 'New problems B' and click on 'others work'
  10. Choose topic 7.2 'The memory inconsistency problem' as the new review topic.
  11. Go ahead and submit a review for this topic.
  12. Now revert back to instructor6 and again go to view review report for this assignment.
  13. Now we can see a new review appears for the student 5059 by the name '5433, student' and this is blue in color indicating that this is yet to be graded.
  14. Now change the earlier grade and click on 'Save'.
  15. Refresh the page and the color of the review '5433, student' changes back to the normal color indicating that this one is graded.

Part 2: Color must not change to blue if instructor has not graded at all .

  1. Follow first 6 steps from Part 1.
  2. This student has submitted one review '5403, student'
  3. The color for this is still the normal color though, as no grading has been done (blank grade and comment column) and so the instructor knows that he has not done any grading for this review.

Part 3: Color must not change to blue if the participant saves a review but does not submit it .

  1. Follow the first 10 steps from part 1.
  2. Now start the review, but do not submit it, just save it.
  3. Revert back to instructor6 and again go to view review report for this assignment.
  4. Now we can see a new review appears for the student 5059 by the name '5433, student' but this still has the normal color as this review is not yet completed and the instructor need not look at it.

Part 4: New Assignments In order to make sure that our changes will also work in the case of newly created assignments, we created a new assignment similar to assignment 'Chapter 11-12 madeup Exercise' under the course CSC/ECE 506, Spring 2015. The participants for this assignment were the same as those for the course. We created 4 new topics for this assignment. We allowed for submissions and reviews to take place at the same time in this assignment. We impersonated users student5695, student5697 and student5698 to make submissions and also made them review each other. After this we tested each of the above three test cases and they were working correctly.

Unit Testing

We have included 2 new helper specs as part of our task

  1. review_mapping_helper_spec.rb
  2. sign_up_sheet_helper_spec.rb

These cover the methods we either modified or added in the respective helpers. We were able to cover the sign_up_sheet_helper entirely while the coverage in case of review_mapping_helper increased from 13.89% to 23.15%

Testing Plan

Testing for modification of dates in staggered deadlines.
Case 1: Attempting to enter a date for a round that was before the previous round. The system should reject this deadline since it is not valid.
Case 2: Attempting to enter a date for round1, round2 or round 3 that is out of invalid (Deadlines being before current date). The system should make sure that all dates being entered are valid and greater than or equal to current date.
Case 3: Entering an incorrect date when manually entering a deadline for a particular round for a staggered deadline. When the user decides to enter a deadline manually rather than choosing the relative deadlines from the dropdown, care must be taken to ensure that the date is in the correct form.

References

Expertiza home page
Github Wikipedia Page
Ruby on Rails Wikipedia Page
Demo Video

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2016_E1708:_Improvements_to_staggered-deadline_assignments&oldid=106601"