CSC/ECE 517 Fall 2017/E1753 OSS project bidding tests
About Expertiza
Expertiza is an open source project developed by North Carolina State University using Ruby on Rails. It is mainly a tool used to collaborate among students and faculty on a course and act as a common repository to track students’ progress on assignments. It is a simple tool where the instructor creates multiple assignments required and teams are assigned projects. Students submit their work and review other’s work and provide feedback.
Wiki link
Github link
Bidding Feature
The bidding feature allows students to sort topics by preference. This is needed in order run the team assignment algorithm, to match students with others based off the similarity in their topic preferences. The feature matches students to teams by calling a team forming algorithm hosted on a web service. Teams are then matched to topics by choosing the most common priority chosen by each member in the team for each topic. Topics are then assigned to students with preference given to the largest team.
• Students submit bids on the sign-up sheet view. The bidding process is done in the lottery controller.
• Bids are saved using the user id or team id based on how the bidding is done with teams getting higher priorities.
• When bids are sent to the sign-up controller, all existing bids for the user’s team are destroyed and replaced by the new bids
• The coloring of the bids is determined by the total number of bids placed on a topic
• One set of bids is possible for entire team. When one team member changes a bid, it will affect the whole team. Currently, each participant has a bid record. A Json request is sent to a webservice hosted on PeerLogic which responds with the new teams. (More information: https://github.com/peerlogic/IntelligentAssignment)
• During topic assignment the teams’ bids are determined by using whichever priority most students placed on a topic (Ex. If 3 students set topic 3 as their 1st priority and 1 set it as their second. Topic 3 would be set as the priority for the team.)
• Matching algorithm: Teams are first sorted by size and a matching algorithm assigns each team to its highest available bid.
Problem Statement
Background
Each team or an individual can go through the list of topics and set priorities for different projects. And once the priorities are set up for all projects, bidding algorithm is executed to assign projects to teams based on the priorities. Also, individuals with similar priorities are combined to form a new team. The following need to be taken care of as part of this work package. (E1753)
• Check that only one signed up team is made for each topic
• Check that users that signed up for the same team stay in the same team
• Should disband teams that are not assigned any topic.
Problems
Problem 1
Existing test cases and Rspec code do not cover the method body statements. Existing test cases only stub a method. Hence code coverage for lottery controller was only at 9%.
Problem 2
Rspec test code was incorrect which resulted in extremely low code coverage. Whole sign up and bidding functionality are present in the controller whereas a good programming in rails consists of all code in models with controllers just for interface applications.
Files Modified
/spec/controllers/lottery_controller_spec.rb
/spec/controllers/sign_up_sheet_controller_spec.rb
Issues and Solutions
Problem 1
Existing test cases for lottery controller was written to test small portion of the function rather than one complete function which is a violation of unit testing.
The test case checks if the web-service is successful in run_intelligent_assignment function. But the function performs other operations along with calling web service.
This rspec test verifies that intelligent assignments are not performed if the is _intelligent variable is set to 0. When the rspec code is executed this test case successfully passes. But when the code is executed by setting the is_intelligent variable to 0, it still executes intelligent assignment.
Problem 2
Code coverage is extremely low and only limited to the function signature even after executing 6 test cases. Screenshot above shows code coverage before implementation of test cases.
Solution
Attached screenshot of code coverage after test cases were written for intelligent assignment method.
As noticed from the screenshot we were unable to cover the last 2 lines of the method since unit testing does not cover JSON.
