CSC/ECE 517 Spring 2015 S1524 FSZZ

E1524. Refactor staggered-deadline assignments

Overview

Introduction to Expertiza

Expertiza is a web application where students can submit and peer-review learning objects (articles, code, web sites, etc). It is used in select courses at NC State and by professors at several other colleges and universities.

Staggered Deadlines

Staggered deadlines involve planning alternate submission dates for papers, projects, or exams when a student has conflicting due dates for these. The key part of staggered deadlines is the planning. Staggered deadlines are always established well in advance of the scheduled due date. It is the advanced planning of these deadlines that makes them "staggered deadlines" rather than extensions<ref>Elmichaels. Difficulties with Staggered Deadlines. Jan 15, 2013</ref>.

Problem Statement

Background

In this semester’s 517 class. Wiki 1a and Wiki 1b are structured as separate assignments, with separate signup sheets, teams, and reviews. But really, since only one of the two was done by any student, it would’ve been better to have a single assignment. Still, some topics could be done soon after the course started, whereas others were better done after we had studied related topics in class.

Staggered-deadline Assignment

This raises the idea of a staggered-deadline assignment, where different topics have different submission and review deadlines, rather than all topics having the same deadline.

Benefit

• Easy to manage. Because in the past, instructor has to build two separate assignments for Wiki 1a and 1b, now one assignment for Wiki is enough.

• Increase flexibility. For instance, when an OSS team facing a much more difficult problem than other teams, instructor may postpone their deadline several days. It will make Expertiza more humanize.

• Take time to do calibration. We find that two Wiki assignments started at Jan. 28th and ended at Feb. 25th, which takes too much time. And with staggered-deadline assignment, we can distribute less time on them. And we can take redundant time to do calibrating. Because we find that many students actually do not know how to evaluate other ones’ work, even there are some rubrics offered. So we recommend to let students grade sample assignments first, then calibrate their grading behaviors. Although it will take some time, we consider it is important to help students distinguish good assignments from common ones.

Product Verion Functionality

• In production version, instructor can set Submission deadline, Review deadline and Metareview deadline for each topic.

Set different deadline for each topic

• Also, instructor can set the dependencies of different topics. And there is a dependency graph generated automatically.

Dependency graph for all topics

Use Case

• Use Case #1: Create staggered-deadline assignment
  • Actor: Instructor
  • Actions:
    • Instructor logs in Expertiza.
    • Instructor creates a new staggered-deadline assignment.

• Use Case #2: Peer review only
  • Actor: Student finish work for first round
  • Actions:
    • Sign up topic.
    • Hand in assignment.
    • Peer review.
    • Write author feedback to reviewer.

• Use Case #3: Peer review and still work on assignments
  • Actor: Student not finish work for first round
  • Actions:
    • Sign up topic.
    • Work on assignment.
    • Peer review.

Use Case Diagram

Error Message Present

• First, login expertiza, in the “Manage Content” page, create a new assignment, for example, the name is “assignment 1b”. Then, as shown below, choose the first icon to edit this assignment.

• Select “Has topics?” and “Staggered deadline assignment?” box in General, and create new topics in Topics, save the properties.

• Back to the “Manage Content” page, this time choose the “Edit signup sheet” button.

• Then, it shows the error: can’t write unknown attribute ‘t_id’.

Puzzle

• If there is more than one round, which means students can submit their assignments more than once, so the staggered-deadline should refer to which deadline?

[Professor]In a staggered-deadline assignment, there is one submission deadline and one review deadline per round. These are always set on a per-topic basis. If the view currently works (and I think it might), as soon as you make an assignment a staggered-deadline assignment, the Signup Sheet (Topics) page will have a link at the bottom to show deadlines for each topic. Click it, and you will see a separate text box for each deadline for each topic.

• If staggered-deadline of one assignment is even later than the second-round review, is it means that that assignment do not need peer review and go directly into meta-review stage?

[Professor]No, there are also separate meta-review deadlines for a staggered-deadline assignment. I think in the current implementation, only submission deadlines, review deadlines, and meta-review deadlines vary by topic. In a more complete implementation, there would be a way for ANY deadline to vary by topic. This would include signup deadlines, drop-topic deadlines, team-formation deadlines, and any other kind of deadline that is defined later.

• When to set staggered-deadline assignments? Create assignment or Before submission?

• What is the meaning of "dependencies" between topics?

References

<references/>