CSC/ECE 517 Fall 2016 E1676 Role-based reviewing

Purpose

In the current version of Expertiza, members among a team can evaluate each other’s contribution and give appropriate scores based on the same questionnaire. However, in the lifetime of software development, members in a team often take on different roles whose works content are vastly differentiated. Like in an agile development environment, such as Scrum, there are typically seven roles group members can take: software engineer, architect, programmer, analyst, QA expert, tester and UI designers . Job description and evaluation entailed are vastly varied among these roles, thus a generic assessment rubric cannot hold reasonably.

The rationale of our project is that we want to give project members the option to evaluate each other's work based on the specific role or duty they take in the development process. This can ensure a reasonable assessment for different duties and can also help to improve the utility of using members-reviewing in the whole reviewing process.

Task Description

Task Description: There are general three aspects we would do in order to achieve the Role-based reviewing function.

1) A new option (check box) would be added to the Review Strategy tab from the instructor view when he creates a new assignment. What’s more, the instructor can create different roles or duties available for the assignment by typing into a text filed. The instructor also has the power to allow multiple members to choose for the same duty.

2) In the Rubrics tab from the instructor view, New rubric would be generated by creating different questionnaires for different roles when adding new assignment.

3) A new option (drop box) would be added to the “Your team” page from the student view when an assignment was created to be role-based reviewing. Student can choose their roles or duties from the drop box which were generated by the instructor for this specific assignment. When the instructor enabled the multiple selection option, different members in the team can choose same duties for this assignment.

Tasks can be viewed more clearly by the graph below:

Design

Database Design

We have modified following three database tables to support role(duty) based reviewing:

1. assignment_questionnaires

2. assignments

3. teams_users

1] We have added following fields in assignment_questionnaires table

Field Name	Type	Description
duty_name	varchar(255)	List of duties that students can choose in the given assignment

2] We have added duty_based, allow_duty_shared and duty_names into assignments table to store the duty name

Field Name	Type	Description
duty_based	boolean	If the assignment's team is duty based, it should be true. If not, it should be false.
allow_duty_share	boolean	If multiple people could choose one duty in the assignment, it should be true. If not, it should be false.
duty_names	varchar(255)	It would save all kinds of duties in the assignment.

3] We have added duty into teams_users table to store the duty.

Field Name	Type	Description
duty	varchar(255)	It would save this user's duty in this team.

Updates

Below are the key files modified:

Controllers

1. expertiza/app/controllers/assignments_controller.rb:
   def edit
       a. add "role based assignment" option to review strategy section.
       b. add roles to current assignment.
       c. add questionnaires to these roles separately. 
   end

2. expertiza/app/controllers/questionnaires_controller.rb

Models

See Database Design section above.

Views

Below are the view files that we modified

1. When logged in as instructor, we modified files below:
 1.1 Add a "role based" strategy checkbox here: expertiza/app/views/assignments/edit/_review_strategy.html.erb
 1.2 Add roles input fields if you check the "role based" strategy checkbox: expertiza/app/views/assignments/edit/_review_strategy.html.erb
 1.3 Create rubric to assign questionnaire based on duties: expertiza-developer/expertiza/app/views/assignments/edit/_rubrics.html.erb
    & expertiza-developer/expertiza/app/helpers/assignment_helper.rb
 
2. When logged in as a student, we modified files below:
 2.1 To show "select duty" and "update duty" option: expertiza/app/views/student_teams/view.html.erb
 2.2 To show the form to select duty: expertiza/app/views/teams_users/edit.html.erb

1] View showing newly added checkbox for setting duty based option and allowing duty share

2] View showing different questionnaire for different teammates based on their duty

3] Assign questionnaire to teammates based on their duty

4] View showing "select duty" link for teammates. Every student can select their duty in the team.

5] After selecting, students can update their duty. Also, they will get the review questionnaire based on duty.

6] Review questionnaire based on duty

Testing Details

In our project, we are providing an option of adding duty for different team members in an assignment and giving review options based on their duty. We will add following functionalities: 1. How instructor create assignment with duties/roles 2. How students take duties from a given list of duties 3. How teammate evaluation rubric assign different review questionnaire based on duty.

UI Testing

UI will be tested manually. First, we will test creating an assignment as an instructor with different duties and assign different review questionnaires to each duty. Then we will login in as a student and take a duty from given list of duties. We will try covering all edge case scenarios in test cases section.

Test Cases

Following will be the test cases:

1. Login in as an Instructor:

 1.a] While creating an assignment, in the Review Strategy tab provide the list of roles/duties possible for teammates. There will be one text box to add      
      duty. To add more duties click on "add more". 
 1.b] Mark the check box that says multiple team members can have the same duty.
 1.c] Go to the Rubrics tab and select different questionnaires for different duty.    
 After creating this assignment students should be able to see different duties in "Your Team" page and should be able to select any duty. Multiple students 
 should be able to select same duty.

2. Login in as an Instructor:

 2.a] Repeat steps 1.a and 1.b 
 2.b] This time do not mark the check box that says "multiple team members can have the same duty".
 After creating this assignment students should be able to see different duties in "Your Team" page. This time student should not be able to select a duty if 
 it was already selected.

3. Login in as a Student

  When you login in as a student you should be able to pick different duties assigned by the instructor. If multiple members are not allowed to have same duty 
  then you should not be able to pick a duty if it was already selected by some other member.

4. Login in as a Reviewer

  When you login in as a reviewer you should get the questionnaire based on your duty. For example, if you are a software engineer you should get the review 
  questionnaire assigned to software engineer. 
  Different roles will be assigned different questionnaires for review.

RSpec

For functional testing, we will use RSpec. It uses Behavior Driven Development (BDD) for testing functionality of the system. It provides way to describe specific functionality of the system being tested and helps in better visualization of test cases.

Capybara

We will use Capybara for automating the manual test cases. It is a web-based automation framework and coupled with RSpec. It allow developers to write the test cases which simulate the whole scenario of manual testing.

Sample Test cases

1. Verify if instructor can select duty option

   it "selects duty based option" do
     login_as("instructor6")
     visit "/assignments/#{@assignment_id}/edit"
     find_link('ReviewStrategy').click
     select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
     fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
     fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
     check("duty_based")
     click_button 'Save'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
                               review_assignment_strategy: 'Auto-Selected',
                               review_topic_threshold: 3,
                               max_reviews_per_submission: 10,
                               duty_based: true
                           )
   end

2. Verify if instructor can correctly set the duty

   it "sets  duty " do
     login_as("instructor6")
     visit "/assignments/#{@assignment_id}/edit"
     find_link('ReviewStrategy').click
     select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
     fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
     fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
     check("duty_based")
     check("allow_duty_share")
     fill_in "assignment_form_assignment_duty_names", with: "developer, tester, analyst, writer, systest"
     click_button 'Save'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
                               review_assignment_strategy: 'Auto-Selected',
                               review_topic_threshold: 3,
                               max_reviews_per_submission: 10,
                               duty_based: true,
                               allow_duty_share: true,
                               duty_names: "developer, tester, analyst, writer, systest"
                           )
    end

3. Verify if duty share is allowed

  it "selects allow duty share option" do
     login_as("instructor6")
     visit "/assignments/#{@assignment_id}/edit"
     find_link('ReviewStrategy').click
     select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
     fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
     fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
     check("duty_based")
     check("allow_duty_share")
     click_button 'Save'
     assignment = Assignment.where(name: 'public assignment for test').first
     expect(assignment).to have_attributes(
                               review_assignment_strategy: 'Auto-Selected',
                               review_topic_threshold: 3,
                               max_reviews_per_submission: 10,
                               duty_based: true,
                               allow_duty_share: true
                           )
   end

4. Verify if student can select duty

   it 'should be able to select duty' do
       login_as @student.name
       click_link @assignment.name
       click_link 'Your team'
       click_link 'select duty'
       select "analyst", from: 'duty'
       click_button 'Save'
       click_on 'Back'
       click_on 'Back'
       page.evaluate_script 'window.location.reload()'
       expect(page).to have_content('analyst')
       expect(page).to have_link('Update duty')
   end

5. Complete test cases are present in following files:

  Student test cases: spec/features/duty_student_spec.rb  [1]
  Instructor test cases: spec/features/assignment_creation_spec.rb [2]

Reference

Expertiza[3]

RSpec [4]

ScrumRoles [5]

CapyBara [6]