CSC/ECE 517 Fall 2017/E1753 OSS project bidding tests: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(22 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==Requirement Change==
==Requirement Change==
Please note that the requirement for the project has been changed to feature tests. The same has been discussed with the professor and the new changes for tests and writeup will be updated once it is done by the re-submission date.
Please note that the requirement for the project has been changed to feature tests (different from the previous review) after discussing with the project mentor and the professor. Updated in Problem statement.


==About Expertiza==
==About Expertiza==
Line 10: Line 10:
===Github link===
===Github link===
*https://github.com/expertiza/expertiza
*https://github.com/expertiza/expertiza


==Bidding Feature==
==Bidding Feature==
Line 19: Line 18:


• Students submit bids on the sign-up sheet view. The bidding process is done in the lottery controller.
• Students submit bids on the sign-up sheet view. The bidding process is done in the lottery controller.
• Bids are saved using the user id or team id based on how the bidding is done with teams getting higher priorities.
• When bids are sent to the sign-up controller, all existing bids for the user’s team are destroyed and replaced by the new bids
• The coloring of the bids is determined by the total number of bids placed on a topic


• One set of bids is possible for entire team. When one team member changes a bid, it will affect the whole team. Currently, each participant has a bid record. A Json request is sent to a webservice hosted on PeerLogic which responds with the new teams.  
• One set of bids is possible for entire team. When one team member changes a bid, it will affect the whole team. Currently, each participant has a bid record. A Json request is sent to a webservice hosted on PeerLogic which responds with the new teams.  
Line 33: Line 26:
• Matching algorithm: Teams are first sorted by size and a matching algorithm assigns each team to its highest available bid.
• Matching algorithm: Teams are first sorted by size and a matching algorithm assigns each team to its highest available bid.


==Problem Statement==


==Problem Statement==
This is an only Testing project to write feature tests for the process. The same is explained in screen shots and video attached. The previous requirement regarding the modifications in sign_up controller have been changed to feature tests and the following problem statements have been taken care of.


===Background===
===Background===
Each team or an individual can go through the list of topics and set priorities for different projects. And once the priorities are set up for all projects, bidding algorithm is executed to assign projects to teams based on the priorities. Also, individuals with similar priorities are combined to form a new team.
Each team or an individual can go through the list of topics and set priorities for different projects. And once the priorities are set up for all projects, bidding algorithm is executed to assign projects to teams based on the priorities.
The following need to be taken care of as part of this work package. (E1753)
The following need to be taken care of as part of this work package. (E1753)


• Check that only one signed up team is made for each topic
• Test for the entirety of the bidding process.


• Check that users that signed up for the same team stay in the same team
• Users are able to place bids on interested topics.


• Should disband teams that are not assigned any topic.
• Users are assigned topics.


==File Modified==
/spec/features/bidding_spec.rb


===Problems===
==Spec Code==


'''Problem 1'''
All bidding process like creation of assignment, adding topics and enabling bidding has to be performed before every test case. This code is placed in the before block of the rspec test code. This code is executed before all the test cases.
[[File:code_before_s.png]]


Existing test cases and Rspec code do not cover the method body statements. Existing test cases only stub a method. Hence code coverage for lottery controller was only at 9%.
For the first test case, code is written to verify the web-service which runs the bidding algorithm is working properly if the bidding process is enabled.  


'''Problem 2'''
[[File:code_success_s.png]]


Rspec test code was incorrect which resulted in extremely low code coverage. Whole sign up and bidding functionality are present in the controller whereas a good programming in rails consists of all code in models with controllers just for interface applications.  
For the second test case, code is written to verify that the bidding process fails if no participant has placed bids.


==Files Modified==
[[File:code_fail.png]]
/spec/controllers/lottery_controller_spec.rb


/spec/controllers/sign_up_sheet_controller_spec.rb
For the third test case, code is written to verify that each user is assigned a topic after running the bidding algorithm.


[[File:code_success_2_s.png]]


==Issues and Solutions==
For the fourth test case, code is written to verify that when the user places multiple bids the bid with the highest priority is assigned to the user.


'''Problem 1'''
[[File:code_success_3_s.png]]


Existing test cases for lottery controller was written to test small portion of the function rather than one complete function which is a violation of unit testing.


==Test Plan==


[[File:partial_test_case.png]]
===Code Coverage===


Code coverage is extremely low and only limited to the function signature even after executing 6 test cases. Screenshot below shows code coverage before implementation of test cases.


The test case checks if the web-service is successful in run_intelligent_assignment function. But the function performs other operations along with calling web service.
[[File:redCoverage.png]]


'''Solution'''


[[File:Wrong_test_case.png]]
Code coverage has drastically increased after the integration test cases are executed. Screenshot below shows code coverage after the execution of the test cases.


This rspec test verifies that intelligent assignments are not performed if the is _intelligent variable is set to 0. When the rspec code is executed this test case successfully passes. But when the code is executed by setting the is_intelligent variable to 0, it still executes intelligent assignment.
[[File:green_coverage_xxsmall.png]]


===Manual Tests===
Manual testing of features on Local host is done with a running Expertiza environment. ( As seen on video). The following steps have been taken care of to setup ad run the bidding process manually.


'''Problem 2'''
• Login as instructor and create assignment. Create topics under the assignment.


[[File:redCoverage.png]]
• Enable bidding and associated properties for the created assignment.
 
• Login as student, create teams and bid for projects.


• Login as instructor and run the bidding algorithm.


Code coverage is extremely low and only limited to the function signature even after executing 6 test cases. Screenshot above shows code coverage before implementation of test cases.
• Login as student and check if topics are assigned


===Automated test cases===


'''Solution'''
The screen shots for the added feature test cases have been attached below. (Also attached as video file). Test cases written to check for the following scenarios.


Attached screenshot of code coverage after test cases were written for intelligent assignment method.
• Check for whole bidding process to run successfully.


[[File:greenCoverage.png]]
• Check for Error message for unsuccessful bidding run.


• Check if user is assigned a topic successfully.


As noticed from the screenshot we were unable to cover the last 2 lines of the method since unit testing does not cover JSON.
• Highest priority bid assigned when multiple bids are placed.


Screenshot of the executed integration tests is attached below.


[[File:testCasePassed.png]]
[[File:Automated_Results_small.png]]

Latest revision as of 07:01, 3 November 2017

Requirement Change

Please note that the requirement for the project has been changed to feature tests (different from the previous review) after discussing with the project mentor and the professor. Updated in Problem statement.

About Expertiza

Expertiza is an open source project developed by North Carolina State University using Ruby on Rails. It is mainly a tool used to collaborate among students and faculty on a course and act as a common repository to track students’ progress on assignments. It is a simple tool where the instructor creates multiple assignments required and teams are assigned projects. Students submit their work and review other’s work and provide feedback.

Wiki link

Github link

Bidding Feature

The bidding feature allows students to sort topics by preference. This is needed in order run the team assignment algorithm, to match students with others based off the similarity in their topic preferences. The feature matches students to teams by calling a team forming algorithm hosted on a web service. Teams are then matched to topics by choosing the most common priority chosen by each member in the team for each topic. Topics are then assigned to students with preference given to the largest team.


• Students submit bids on the sign-up sheet view. The bidding process is done in the lottery controller.

• One set of bids is possible for entire team. When one team member changes a bid, it will affect the whole team. Currently, each participant has a bid record. A Json request is sent to a webservice hosted on PeerLogic which responds with the new teams. (More information: https://github.com/peerlogic/IntelligentAssignment)

• During topic assignment the teams’ bids are determined by using whichever priority most students placed on a topic (Ex. If 3 students set topic 3 as their 1st priority and 1 set it as their second. Topic 3 would be set as the priority for the team.)

• Matching algorithm: Teams are first sorted by size and a matching algorithm assigns each team to its highest available bid.

Problem Statement

This is an only Testing project to write feature tests for the process. The same is explained in screen shots and video attached. The previous requirement regarding the modifications in sign_up controller have been changed to feature tests and the following problem statements have been taken care of.

Background

Each team or an individual can go through the list of topics and set priorities for different projects. And once the priorities are set up for all projects, bidding algorithm is executed to assign projects to teams based on the priorities. The following need to be taken care of as part of this work package. (E1753)

• Test for the entirety of the bidding process.

• Users are able to place bids on interested topics.

• Users are assigned topics.

File Modified

/spec/features/bidding_spec.rb

Spec Code

All bidding process like creation of assignment, adding topics and enabling bidding has to be performed before every test case. This code is placed in the before block of the rspec test code. This code is executed before all the test cases.

For the first test case, code is written to verify the web-service which runs the bidding algorithm is working properly if the bidding process is enabled.

For the second test case, code is written to verify that the bidding process fails if no participant has placed bids.

For the third test case, code is written to verify that each user is assigned a topic after running the bidding algorithm.

For the fourth test case, code is written to verify that when the user places multiple bids the bid with the highest priority is assigned to the user.


Test Plan

Code Coverage

Code coverage is extremely low and only limited to the function signature even after executing 6 test cases. Screenshot below shows code coverage before implementation of test cases.

Solution

Code coverage has drastically increased after the integration test cases are executed. Screenshot below shows code coverage after the execution of the test cases.

Manual Tests

Manual testing of features on Local host is done with a running Expertiza environment. ( As seen on video). The following steps have been taken care of to setup ad run the bidding process manually.

• Login as instructor and create assignment. Create topics under the assignment.

• Enable bidding and associated properties for the created assignment.

• Login as student, create teams and bid for projects.

• Login as instructor and run the bidding algorithm.

• Login as student and check if topics are assigned

Automated test cases

The screen shots for the added feature test cases have been attached below. (Also attached as video file). Test cases written to check for the following scenarios.

• Check for whole bidding process to run successfully.

• Check for Error message for unsuccessful bidding run.

• Check if user is assigned a topic successfully.

• Highest priority bid assigned when multiple bids are placed.

Screenshot of the executed integration tests is attached below.

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2017/E1753_OSS_project_bidding_tests&oldid=111748"