CSC/ECE 517 Fall 2016 E1702 Lightweight badging system based on Credly

Purpose

The purpose of the project is to acknowledge the accomplishments of best performers in the class on various pre-decided areas. It will also motivate other students to perform better academically. It is deduced from a research at the University of Chicago which shows that Test performance can improve dramatically if students are offered rewards just before they are given standardized tests and if they receive the incentive afterward.

Problem Statement

The aim of this project is to implement a badging system for expertiza where certain students are awarded with badges based on their exceptional academic performance in assignments, project submissions etc. We have used Credly to make Badges. We are asked to make four badges now. And we can always add, remove or modify the existing badges in the future. The four badges selected by the instructor Zhewei Hu and our Team are:

1 Top Score

2 Dream Team

3 Good Reviewer

4 Consistency

Detailed Description

LIGHTWEIGHT BADGING SYSTEM in Expertiza will act as a rewarding module of Expertiza to the students where they will be awarded badges, which can be understood as medals , in addition to the scores earned. We have decided on four badges which will be awarded to students who fulfil certain conditions explained in detail below.

Top Score

Icon

Who will be Awarded

For every assignment in Expertiza, we have students perform differently on a scale of 100 based on the reviews their projects receive from their peers. The topper badges will be used to honor the team(s) which scores the maximum for a assignment.

For example:

If we take the same example, the following Team will win the Top Score reward for OSS assignment

Rationale

The motive behind choosing this badge is to honor the hard work of team(s) who scored the maximum in class. This way, we can motivate other students to score better in the next assignment and score the best. We wish to achieve better quality submissions by introducing this badge.

Dream Team

Icon

Who will be Awarded

In Expertiza students are required to give reviews to the team mates they worked with for a project. This is evaluated on a scale of 100%. The Dream Team badge will be awarded to the team(s) whose each member received at least 95% average score in the teammate reviews for a particular project.

For example:

If we take the same example, Team with topic E1324 will win the Dream Team badge as each member got at least 95% in teammate review score for OSS assignment.

Rationale

The motive behind choosing this badge is that sometimes it is observed that only a few students from a team work. This is mainly due to little or no penalty for doing less work in a Team or not doing ones part of the work. Though the efficient teammates gives low review scores to the not working partners, it does little to motivate the later to contribute equally. To prevent this, the badge will encourage students get good teammate reviews for which they will have to contribute efficiently. This way we think that all the team members will have an incentive to do their parts.

Good Reviewer

Icon

Who will be Awarded

In Expertiza students are required to give reviews to the work of other teams. This is evaluated on a scale of 100% by the teaching staff. The Good Reviewer badge will be awarded to the student(s) who receive at least 95% score in the peer reviews awarded by the teaching staff for a project.

For example:

If we take the same example, the following students will win the Good Reviewer badge as each of them got at least 95% in peer review score for OSS assignment

Rationale

Good reviewer badge will motivate students to give good reviews to fellow students on their work. This will serve dual purpose. First, It will help the students to improve their work taking advice from healthy feedback. Second, it will motivate the student to give good reviews and go through the work of other students which will give him/her the opportunity to learn.

Consistency

Icon

Who will be Awarded

The performance of students across assignments vary. The Consistency badge will be awarded to the student(s) who receive at least 90% score in all the assignments. This badge will help the instructor also to know the consistent performers of the class.

For example:

If we take the same example, the following students will win the Consistency badge as each of them got at least 90% in Assignment score for all the assignments.

Rationale

The motive behind choosing this badge is to motivate students to perform consistently throughout the semester. This will help keep students on their toes throughout the semester and result in better quality submissions. It will also improve the overall grade of the class.

Implementation in Expertiza

Test Plan

We have manually tested the badges. We have logged in as an instructor6 and identified the top scorer of a particular assignment, a Team in which each member has got more than or equal to 95%, students who has got good review score from instructor and has consistently secured above 90% in all the assignments. We have also identified a set of students who did not follow the mentioned criteria.

Next, we impersonated each of the student to verify that only the badges they deserve is displayed on the homepage, below the corresponding assignment name. We further checked the same thing for instructors by visiting the participant_list/list view page.

The link for the demo video is :

You can also replicate the testing according to our video.

You can find the pull request in here : Pull Request

Future Improvements

Currently, the average score of all the reviews for each assignment is computed dynamically and not stored anywhere in the database. This incurs huge latency in loading the pages which queries the score. This is the reason why "Review Score" page from instructor view takes long time to load.

For the topper badge, we need to query the average score of all the teams for all the assignments in a particular course and decide whether the current user is the member of the team having the highest score in any of these assignments. This involves large number of computation and increases the home page load time of student view by an unacceptably large margin. Currently, a project is underway to fix this problem by storing the scores of teams in database. Once this task is completed, we can refactor our Top Score method to take advantage of this change. This will significantly reduce the latency of the Top Score badge for both student and Instructor view.