CSC/ECE 517 Fall 2016 E1676 Role-based reviewing
Purpose
In the current version of Expertiza, members among a team can evaluate each other’s contribution and give appropriate scores based on the same questionnaire. However, in the lifetime of software development, members in a team often take on different roles whose works content are vastly differentiated. Like in an agile development environment, such as Scrum, there are typically seven roles group members can take: software engineer, architect, programmer, analyst, QA expert, tester and UI designers . Job description and evaluation entailed are vastly varied among these roles, thus a generic assessment rubric cannot hold reasonably.
The rationale of our project is that we want to give project members the option to evaluate each other's work based on the specific role or duty they take in the development process. This can ensure a reasonable assessment for different duties and can also help to improve the utility of using members-reviewing in the whole reviewing process.
Task Description
Task Description: There are general three aspects we would do in order to achieve the Role-based reviewing function.
1) A new option (check box) would be added to the Review Strategy tab from the instructor view when he creates a new assignment. What’s more, the instructor can create different roles or duties available for the assignment by typing into a text filed. The instructor also has the power to allow multiple members to choose for the same duty.
2) In the Rubrics tab from the instructor view, New rubric would be generated by creating different questionnaires for different roles when adding new assignment.
3) A new option (drop box) would be added to the “Your team” page from the student view when an assignment was created to be role-based reviewing. Student can choose their roles or duties from the drop box which were generated by the instructor for this specific assignment. When the instructor enabled the multiple selection option, different members in the team can choose same duties for this assignment.
Tasks can be viewed more clearly by the graph below:
Design
Database Design
We have modified following three database tables to support role(duty) based reviewing:
1. assignment_questionnaires
2. assignments
3. teams_users
| Field Name | Type | Description |
|---|---|---|
| duty_name | varchar(255) | List of duties that students can choose in the given assignment |
| Field Name | Type | Description |
|---|---|---|
| duty_based | boolean | If the assignment's team is duty based, it should be true. If not, it should be false. |
| allow_duty_share | boolean | If multiple people could choose one duty in the assignment, it should be true. If not, it should be false. |
| duty_names | varchar(255) | It would save all kinds of duties in the assignment. |
| Field Name | Type | Description |
|---|---|---|
| duty | varchar(255) | It would save this user's duty in this team. |
Updates
Below are the key files modified:
Controllers
1. expertiza/app/controllers/assignments_controller.rb:
def edit
a. add "role based assignment" option to review strategy section.
b. add roles to current assignment.
c. add questionnaires to these roles separately.
end
2. expertiza/app/controllers/questionnaires_controller.rb
Models
See Database Design section above.
Views
Below are the view files that we modified
1. When logged in as instructor, we modified files below:
1.1 Add a "role based" strategy checkbox here: expertiza/app/views/assignments/edit/_review_strategy.html.erb
1.2 Add roles input fields if you check the "role based" strategy checkbox: expertiza/app/views/assignments/edit/_review_strategy.html.erb
1.3 Create rubric to assign questionnaire based on duties: expertiza-developer/expertiza/app/views/assignments/edit/_rubrics.html.erb
& expertiza-developer/expertiza/app/helpers/assignment_helper.rb
2. When logged in as a student, we modified files below:
2.1 To show "select duty" and "update duty" option: expertiza/app/views/student_teams/view.html.erb
2.2 To show the form to select duty: expertiza/app/views/teams_users/edit.html.erb
1] View showing newly added checkbox for setting duty based option and allowing duty share
2] View showing different questionnaire for different teammates based on their duty
3] Assign questionnaire to teammates based on their duty
4] View showing "select duty" link for teammates. Every student can select their duty in the team.
5] After selecting, students can update their duty. Also, they will get the review questionnaire based on duty.
6] Review questionnaire based on duty
Testing Details
In our project, we are providing an option of adding duty for different team members in an assignment and giving review options based on their duty. We will add following functionalities: 1. How instructor create assignment with duties/roles 2. How students take duties from a given list of duties 3. How teammate evaluation rubric assign different review questionnaire based on duty.
UI Testing
UI will be tested manually. First, we will test creating an assignment as an instructor with different duties and assign different review questionnaires to each duty. Then we will login in as a student and take a duty from given list of duties. We will try covering all edge case scenarios in test cases section.
Test Cases
Following will be the test cases:
1. Login in as an Instructor:
1.a] While creating an assignment, in the Review Strategy tab provide the list of roles/duties possible for teammates. There will be one text box to add
duty. To add more duties click on "add more".
1.b] Mark the check box that says multiple team members can have the same duty.
1.c] Go to the Rubrics tab and select different questionnaires for different duty.
After creating this assignment students should be able to see different duties in "Your Team" page and should be able to select any duty. Multiple students
should be able to select same duty.
2. Login in as an Instructor:
2.a] Repeat steps 1.a and 1.b 2.b] This time do not mark the check box that says "multiple team members can have the same duty". After creating this assignment students should be able to see different duties in "Your Team" page. This time student should not be able to select a duty if it was already selected.
3. Login in as a Student
When you login in as a student you should be able to pick different duties assigned by the instructor. If multiple members are not allowed to have same duty then you should not be able to pick a duty if it was already selected by some other member.
4. Login in as a Reviewer
When you login in as a reviewer you should get the questionnaire based on your duty. For example, if you are a software engineer you should get the review questionnaire assigned to software engineer. Different roles will be assigned different questionnaires for review.
RSpec
For functional testing, we will use RSpec. It uses Behavior Driven Development (BDD) for testing functionality of the system. It provides way to describe specific functionality of the system being tested and helps in better visualization of test cases.
Capybara
We will use Capybara for automating the manual test cases. It is a web-based automation framework and coupled with RSpec. It allow developers to write the test cases which simulate the whole scenario of manual testing.
Sample Test cases
1. Verify if instructor can select duty option
it "selects duty based option" do
login_as("instructor6")
visit "/assignments/#{@assignment_id}/edit"
find_link('ReviewStrategy').click
select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
check("duty_based")
click_button 'Save'
assignment = Assignment.where(name: 'public assignment for test').first
expect(assignment).to have_attributes(
review_assignment_strategy: 'Auto-Selected',
review_topic_threshold: 3,
max_reviews_per_submission: 10,
duty_based: true
)
end
2. Verify if instructor can correctly set the duty
it "sets duty " do
login_as("instructor6")
visit "/assignments/#{@assignment_id}/edit"
find_link('ReviewStrategy').click
select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
check("duty_based")
check("allow_duty_share")
fill_in "assignment_form_assignment_duty_names", with: "developer, tester, analyst, writer, systest"
click_button 'Save'
assignment = Assignment.where(name: 'public assignment for test').first
expect(assignment).to have_attributes(
review_assignment_strategy: 'Auto-Selected',
review_topic_threshold: 3,
max_reviews_per_submission: 10,
duty_based: true,
allow_duty_share: true,
duty_names: "developer, tester, analyst, writer, systest"
)
end
3. Verify if duty share is allowed
it "selects allow duty share option" do
login_as("instructor6")
visit "/assignments/#{@assignment_id}/edit"
find_link('ReviewStrategy').click
select "Auto-Selected", from: 'assignment_form_assignment_review_assignment_strategy'
fill_in 'assignment_form_assignment_review_topic_threshold', with: 3
fill_in 'assignment_form_assignment_max_reviews_per_submission', with: 10
check("duty_based")
check("allow_duty_share")
click_button 'Save'
assignment = Assignment.where(name: 'public assignment for test').first
expect(assignment).to have_attributes(
review_assignment_strategy: 'Auto-Selected',
review_topic_threshold: 3,
max_reviews_per_submission: 10,
duty_based: true,
allow_duty_share: true
)
end
4. Verify if student can select duty
it 'should be able to select duty' do
login_as @student.name
click_link @assignment.name
click_link 'Your team'
click_link 'select duty'
select "analyst", from: 'duty'
click_button 'Save'
click_on 'Back'
click_on 'Back'
page.evaluate_script 'window.location.reload()'
expect(page).to have_content('analyst')
expect(page).to have_link('Update duty')
end
5. Complete test cases are present in following files:
Student test cases: spec/features/duty_student_spec.rb [1] Instructor test cases: spec/features/assignment_creation_spec.rb [2]
Reference
Expertiza[3]
RSpec [4]
ScrumRoles [5]
CapyBara [6]