CSC/ECE 517 Fall 2016/E1701. Accelerate RSpec testing

Expertiza

Expertiza is a web application where students can submit and peer-review learning objects (articles, code, web sites, etc). It is used in select courses at NC State and by professors at several other colleges and universities.

Rspec

Rspec is a meta-gem, which depends on the rspec-core, rspec-expectations and rspec-mocks gems. Each of these can be installed separately and loaded in isolation using require. Among other benefits, this allows you to use rspec-expectations, for example, in Test::Unit::TestCase if you happen to prefer that style. Conversely, if you like RSpec's approach to declaring example groups and examples (describe and it) but prefer Test::Unit assertions and mocha, rr or flexmock for mocking, you'll be able to do that without having to install or load the components of RSpec that you're not using.

Problem Statement

Unfortunately, Expertiza tests are really slow. If you check the TravisCI, it needs more than 9 min to run all tests.

One reason is that we use fixture to create records in test DB each time running tests.

For example, the codes in quiz_spec.rb contain too much create so that prolong testing time.

   # Create an assignment due date
    create(:deadline_type, name: "submission")
    create(:deadline_type, name: "review")
    create(:deadline_type, name: "metareview")
    create(:deadline_type, name: "drop_topic")
    create(:deadline_type, name: "signup")
    create(:deadline_type, name: "team_formation")
    create(:deadline_right)
    create(:deadline_right, name: 'Late')
    create(:deadline_right, name: 'OK')
    create :assignment_due_date, due_at: (DateTime.now + 1)

One solution is building an complete database to support all RSpec test without creating new records.

Task

Formally, we need to:

• Create the records in test DB according to the content in fixtures.
• Check each test file and delete certain DB records creation code that insert default records (eg. create(:deadline_type)).
• And keep all the test cases passing when using test DB and make sure the time running test cases is shorter than before.
• You should submit the sql file of test DB to Expertiza.

To finish these tasks, we need to modify all RSpec test files in Expertiza and using a new Database:Expertiza_test.

Refactor files

test database

Rspec tests

Design

Database design

We design a expertiza_test database to save the test date used for Rspec. The test database owns the same structure as the real expertiza database, including the relations between tables and some restriction of attribute like it cannot be null or some other requirements for different attributes. Besides, the data in test database is the same as the data in factories part in spec, which includes FeedbackResponseMap.rb, Respone.rb, factories.rb, quiz_factory.rb. In this situation we do not need to create some data before testing, and we can use the data in test DB directly. Because of it, the overhead of testing cases will experience obvious decreasing. like the Rspec statements in creating some User objects.:

 factory :admin, class: User do
   sequence(:name) {|n| "admin#{n}" }
   role { Role.where(name: 'Administrator').first || association(:role_of_administrator) }
   password "password"
   password_confirmation "password"
   sequence(:fullname) {|n| "#{n}, administrator" }
   email "expertiza@mailinator.com"
   parent_id 1
   private_by_default  false
   mru_directory_path  nil
   email_on_review true
   email_on_submission true
   email_on_review_of_review true
   is_new_user false
   master_permission_granted 0
   handle "handle"
   leaderboard_privacy false
   digital_certificate nil
   timezonepref nil
   public_key nil
   copy_of_emails  false
 end

then transfer this into SQL query and then add the information the same as designed for Rspec test into expertiza_test database.

We will try two method to achieve the goal:

1. Write a script to create all test data automatically then don't clean the database.
2. Build the database manually using SQL:

 INSERT INTO User VALUES (xxxxx, xxxxxx, xxxxx, xxxxxxxx, xxxxxxx, xxxxxxx, xxxx );

Feature test design

To accelerate the feature test, we have to eliminate pseudo data creation. Now all the feature tests must create their own data before each test. For an example, now a test first create a user before it can do the login operation. The creation latency is considerable. After building a test database, all the pseudo data creation statement can be removed from the test files. Instead, the tests reference data stored in the test database. This change will not only accelerate the testing time, but also dry out the code.

The way tests reference data from database is the same as it does in the development mode. To login, or do other operations, the test can invoke existing user data in test database. Using "User.find()" or "User.where()", test data can easily be invoked by tests.

For existing tests, they can directly use the test database. For new tests added later, they can use the records in test database or create specialized data use FactoryGirl. So to keep the test database up to date ,it needs to be maintained regularly.