CSC/ECE 517 Fall 2016 E1702 Lightweight badging system based on Credly
Purpose
The purpose of the project is to acknowledge the accomplishments of best performers in the class on various pre-decided areas. It will also motivate other students to perform better academically. It is deduced from a research at the University of Chicago which shows that Test performance can improve dramatically if students are offered rewards just before they are given standardized tests and if they receive the incentive afterward.
Problem Statement
The aim of this project is to implement a badging system for expertiza where certain students are awarded with badges based on their exceptional academic performance in assignments, project submissions etc. We have used Credly to make Badges. We are asked to make four badges now. And we can always add, remove or modify the existing badges in the future. The four badges selected by the instructor Zhewei Hu and our Team are:
1 Top Score
2 Dream Team
3 Good Reviewer
4 Consistency
Detailed Description
LIGHTWEIGHT BADGING SYSTEM in Expertiza will act as a rewarding module of Expertiza to the students where they will be awarded badges, which can be understood as medals , in addition to the scores earned. We have decided on four badges which will be awarded to students who fulfil certain conditions explained in detail below.
Top Score
Icon
Who will be Awarded
For every assignment in Expertiza, we have students perform differently on a scale of 100 based on the reviews their projects receive from their peers. The topper badges will be used to honor the team(s) which scores the maximum for a assignment.
For example:
If we take the same example, the following Team will win the Top Score reward for OSS assignment
Rationale
The motive behind choosing this badge is to honor the hard work of team(s) who scored the maximum in class. This way, we can motivate other students to score better in the next assignment and score the best. We wish to achieve better quality submissions by introducing this badge.
Dream Team
Icon
Who will be Awarded
In Expertiza students are required to give reviews to the team mates they worked with for a project. This is evaluated on a scale of 100%. The Dream Team badge will be awarded to the team(s) whose each member received at least 95% average score in the teammate reviews for a particular project.
For example:
If we take the same example, Team with topic E1324 will win the Dream Team badge as each member got at least 95% in teammate review score for OSS assignment.
Rationale
The motive behind choosing this badge is that sometimes it is observed that only a few students from a team work. This is mainly due to little or no penalty for doing less work in a Team or not doing ones part of the work. Though the efficient teammates gives low review scores to the not working partners, it does little to motivate the later to contribute equally. To prevent this, the badge will encourage students get good teammate reviews for which they will have to contribute efficiently. This way we think that all the team members will have an incentive to do their parts.
Good Reviewer
Icon
Who will be Awarded
In Expertiza students are required to give reviews to the work of other teams. This is evaluated on a scale of 100% by the teaching staff. The Good Reviewer badge will be awarded to the student(s) who receive at least 95% score in the peer reviews awarded by the teaching staff for a project.
For example:
If we take the same example, the following students will win the Good Reviewer badge as each of them got at least 95% in peer review score for OSS assignment
Rationale
Good reviewer badge will motivate students to give good reviews to fellow students on their work. This will serve dual purpose. First, It will help the students to improve their work taking advice from healthy feedback. Second, it will motivate the student to give good reviews and go through the work of other students which will give him/her the opportunity to learn.
Consistency
Icon
Who will be Awarded
The performance of students across assignments vary. The Consistency badge will be awarded to the student(s) who receive at least 90% score in all the assignments. This badge will help the instructor also to know the consistent performers of the class.
For example:
If we take the same example, the following students will win the Consistency badge as each of them got at least 90% in Assignment score for all the assignments.
Rationale
The motive behind choosing this badge is to motivate students to perform consistently throughout the semester. This will help keep students on their toes throughout the semester and result in better quality submissions. It will also improve the overall grade of the class.
How it looks like
Student View
This is how the student page will look like.
Instructor View
This is how the instructor page will look like.
Implementation in Expertiza
The above diagram shows the implementation of badges for students.
1. The badge.rb is the Model class. It will contain the business logic for various badges.
2. The student_task_controller.rb is the controller class which will use the methods of badge.rb to determine which badges a particular individual/team has earned.
3. The list.html.erb in in student_task folder is the view class which will use the data to display the badges to the user.
The above diagram shows the implementation of badges for instructors. The data flow is same as that of students. The badge.rb is the model class, participants_controller.rb is the controller class and list.html.erb under participants folder is the view class. Their purpose is same as their corresponding colleagues in previous implementation.
Rationale behind Implementation
1. As per requirement, the badges need to be displayed in the list view for both instructors and students. And thus, the list.html.erb has been chosen as the views to be modified.
2. The controllers student_task_controller.rb and participants_controller.rb are invoking the views mentioned in the above point. Thus, this list method in both the controllers need to be refactored to figure out what badges the team/ individual deserves.
3. All business logic regarding badge assignment criteria will be put in badge.rb, a model class created for the specific purpose.
4. Both the view and the controller mentioned above will have minor refactoring.
5. Since all business logic will be in the model class this class can be easily expanded to include more badges in the future. This provides both modularity and scalability which are the most important consideration behind the design.
Test Plan
Controller Test
There were no existing test script for student_task controller and participants controller. We created two test scripts (spec/controllers/student_task_controller_spec.rb and (spec/controllers/participants_controller_spec.rb) to automatically test the list method in both of the controllers. The scenarios we tested are explained below:
1: If you try to access the student_task/list or participants/list when you are not logged in, you will be redirected to welcome page ("\").
2: You can visit student_task/list by logging in as either a student or an instructor by invoking the list method in student_task controller.
3: You can visit participants/list by logging in as an instructor by invoking the list method in participants controller.
4: If you try to access the participants/list when you are logged in as a student, you will be redirected to welcome page ("\").
We used the two scripts initially to make sure that the list method in both the controllers are working as expected before we started adding any code. Later, when we were adding code for the required functionality , we used these scripts for regression testing.
Manual Test
We have thoroughly tested the all the functionalities we have implemented by manually testing expertiza.
1. We have logged in as instructor6 and identified the top scorer of a particular assignment. Next, we impersonated all the students of the top score team to make sure all of them have received the top score badge for that particular assignment.
2. A Team in which each member has got more than or equal to 95% from all team mates review should receive the dream team badge. This has been verified by logging in as the members of the team.
3. We have logged in as instructor and given 100 points and 90 points to 2 students respectively as review scores for a particular assignment. Then we impersonated the 2 students to check that one of them has received a good reviewer badge and the other hasn't received the badge for that assignment.
4. Logging in as instructor, we have identified a student who has scored more than or equal to 90% in all the assignment for that course. Next we logged in as that student to make sure (s)he has received the consistency badge.
We have further checked that instructors can see the badges the student has received for a particular assignment by visiting the participant_list/list view page. The page shows all the assignment specific badges (Top score, Dream team and Good reviewer badge).
The link for the demo video is : Demo Video
You can also replicate the testing according to our video.
You can find the pull request in here : Pull Request
Future Improvements
Currently, the average score of all the reviews for each assignment is computed dynamically and not stored anywhere in the database. This incurs huge latency in loading the pages which queries the score. This is the reason why "Review Score" page from instructor view takes long time to load.
For the topper badge, we need to query the average score of all the teams for all the assignments in a particular course and decide whether the current user is the member of the team having the highest score in any of these assignments. This involves large number of computation and increases the home page load time of student view by an unacceptably large margin. Currently, a project is underway to fix this problem by storing the scores of teams in database. Once this task is completed, we can refactor our Top Score method to take advantage of this change. This will significantly reduce the latency of the Top Score badge for both student and Instructor view.