CSC/ECE 517 Fall 2018/E1873 Specialized rubics for different topic types: Difference between revisions
(→Result) |
|||
(67 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
= Introduction = | ==''' Introduction '''== | ||
== Background == | === Background === | ||
One of fantastic features that Expertiza provides is for each assignment, it allows students providing reviews for others' jobs. This is done by offering a pre-defined questionnaire for the whole assignment, which is called the rubric for this assignment. Each rubric has several questions, each question provides a criterion to evaluate the performance from a specific angle. Then students can evaluate the entire job by rating and | One of the fantastic features that Expertiza provides is for each assignment, it allows students providing reviews for others' jobs. This is done by offering a pre-defined questionnaire for the whole assignment, which is called the rubric for this assignment. Each rubric has several questions, each question provides a criterion to evaluate the performance from a specific angle. Then students can evaluate the entire job by rating and answering these questions. The team did that job can also get a detailed feedback for different aspects of the project. | ||
== Motivation == | === Motivation === | ||
One of the problems for the review system is that for each assignment, different topics are included. Students can choose the one topic they like, and those topics fall into different categories. That is, an assignment always include different kind of projects. In Expertiza, the topic can be refactoring projects, testing projects, Mozilla projects, etc. Apparently, for different kinds of projects, we have different criteria to evaluate their performance. However, due to the current design of the system, all projects belong to the same assignment can only have the same rubric, which may lead to unnecessary criteria for some projects and inadequate criteria for other projects. | |||
To make sure all kinds of projects can be evaluated properly. We plan to refactor the system to allow | To make sure all kinds of projects can be evaluated properly. We plan to refactor the system to allow assigning different rubies for different projects in a specified appointment. | ||
==''' Analysis & Plan '''== | |||
To realize such purpose, we have the following tasks need to be done: | To realize such purpose, we have the following tasks need to be done: | ||
===Check Box=== | |||
For current design, the rubric used for a specified assignment is set at the following page. To make sure users can choose to set different rubrics for different projects, we need first add a checkbox at this page. Therefore, users can use this box to indicate if they want to use the same rubric for all the projects or set some different rubrics for some projects. | |||
[[File:2018F1873rubric.png]] | [[File:2018F1873rubric.png]] | ||
===Dropdown List=== | |||
To make sure users can choose rubrics for each project, we should add a new dropdown list beside each project in the following page. The number of dropdown list should decide by review round number. Therefore, each time users choose to identify different rubrics for different projects, the list will appear and users can choose proper rubrics. The default rubric set at last page will be overwritten in this case. | |||
[[File:2018F1873project.png]] | [[File:2018F1873project.png]] | ||
===Controller=== | |||
Make changes to the controller so that these functions can work. Most of our logic control codes will be written in ''helpers/sign_up_sheet_helper.rb''. | |||
logic flowchart: | |||
[[File:umlj.png]] | |||
== | ===Database=== | ||
Add questionnaire_id in sign_up_topics table to make sure the projects can always relate to correct rubrics | |||
[[File:379jjj.png]] | |||
===Test=== | |||
Test what we changed to make sure it works as intended and not influence the original system. | |||
==''' Implement Steps '''== | |||
=== Task 1 Add checkbox=== | === Task 1 Add checkbox=== | ||
1. edit the file views/assignments/edit/_rubrics.html.erb | |||
2. add a checkbox at the top of the page which says "Vary rubric by topic" | 2. add a checkbox at the top of the page which says "Vary rubric by topic" | ||
3. set the default state for this checkbox as unchecked | |||
3. set the default state for this checkbox as unchecked | |||
===Task 2 Add dropdown list=== | ===Task 2 Add dropdown list=== | ||
1. edit the file views/assignment/edit/_general.html.erb | |||
2. add a variable to indicate if the checkbox said in Task 1 is checked | 2. add a variable to indicate if the checkbox said in Task 1 is checked | ||
3. create a new file under views/sign_up_sheet called "_rubrics_list.html.erb" to store the code for the dropdown list | 3. create a new file under views/sign_up_sheet called "_rubrics_list.html.erb" to store the code for the dropdown list | ||
4. edit the file views/sign_up_sheet/_table_line.html.erb | 4. edit the file views/sign_up_sheet/_table_line.html.erb | ||
5. add the code to render the dropdown list | 5. add the code to render the dropdown list | ||
6. making a judgment before | |||
6. making a judgment before rendering the dropdown list, only if the checkbox in Task 1ist checked to render the page | |||
===Task 3 Change controllers=== | ===Task 3 Change controllers=== | ||
Change the update method in sign_up_sheet_controller.rb to make sure when new rubric is chosen for a specific topic, it can be stored in the database properly | |||
===Task 4 Create migration=== | ===Task 4 Create migration=== | ||
1. create a new migration to add questionnaire_id in sign_up_topics | |||
2. the questionnaire_id is a foreign key which references to questionnaires | 2. the questionnaire_id is a foreign key which references to questionnaires | ||
3. change the file models/sign_up_topic.rb, and add the relations with questionnaire, the relation between them is one-to-one | |||
3. change the file models/sign_up_topic.rb, and add the relations with the questionnaire, the relation between them is one-to-one | |||
===Task 5 Test the changes=== | ===Task 5 Test the changes=== | ||
1. make a test plan for the features that need to be tested | |||
2. follow the test plan to write automatic test codes | 2. follow the test plan to write automatic test codes | ||
3. evaluate the results and refactor codes</pre> | |||
3. evaluate the results and refactor codes | |||
==''' Test plan '''== | |||
We mainly use cucumber and capybara to test UI, and the following are detailed plans | |||
===Test checkbox=== | |||
1. go to the rubrics page under the assignment | |||
2. we can see the checkbox says "Vary rubric by topic" shows on this page | |||
3. the initial state of the checkbox is unchecked | |||
===Test dropdown list=== | |||
====Scenario 1==== | |||
1. the checkbox "Vary rubric by topic" is unchecked | |||
2. go to the topics page under assignment | |||
3.there are no dropdown lists beside topics | |||
====Scenario 2==== | |||
1. the checkbox "Vary rubric by topic" is checked | |||
2. go to the topics page under assignment | |||
3. the table to show the topics have a row called "rubrics" | |||
4. under this title, each topic has a dropdown list | |||
==''' Result '''== | |||
[[File:Www11.png]] | |||
[[File:Jjj259.png]] | |||
===Test=== | |||
Instead of testing UI, we did some unit tests using Rspec. | |||
For testing update_topic_questionnaires. | |||
*when attributes are nil | |||
<pre> | |||
it 'returns false' do | |||
expect(assignment_form.update_topic_questionnaires(nil)).to be false | |||
end | |||
</pre> | |||
*when attributes are not nil and at least one topic_questionnaire's id is nil or blank | |||
<pre> | |||
let(:attributes) { [topic_questionnaire, topic_questionnaire2] } | |||
before(:each) do | |||
allow(topic_questionnaire).to receive(:[]).with(:id).and_return(nil) | |||
allow(topic_questionnaire2).to receive(:[]).with(:id).and_return(1) | |||
allow(TopicQuestionnaire).to receive(:where).with(sign_up_topic_id: SignUpTopic.where(assignment_id: 1)).and_return([]) | |||
allow(TopicQuestionnaire).to receive(:new).with(topic_questionnaire).and_return(topic_questionnaire) | |||
allow(TopicQuestionnaire).to receive(:find).with(1).and_return(topic_questionnaire2) | |||
end | |||
</pre> | |||
*when both save and update_attributes method do not work | |||
<pre> | |||
it 'changes @has_errors value to true and returns attributes (args)' do | |||
allow(topic_questionnaire).to receive(:save).and_return(false) | |||
allow(topic_questionnaire2).to receive(:update_attributes).with(topic_questionnaire2).and_return(false) | |||
expect(assignment_form.update_topic_questionnaires(attributes)).to eq(attributes) | |||
expect(assignment_form.instance_variable_get(:@has_errors)).to be true | |||
end | |||
</pre> | |||
*when both save and update_attributes method work well | |||
<pre> | |||
it 'returns attributes (args) and @has_errors value is nil' do | |||
allow(topic_questionnaire).to receive(:save).and_return(true) | |||
allow(topic_questionnaire2).to receive(:update_attributes).with(topic_questionnaire2).and_return(true) | |||
expect(assignment_form.update_topic_questionnaires(attributes)).to eq(attributes) | |||
expect(assignment_form.instance_variable_get(:@has_errors)).to be nil | |||
end | |||
</pre> | |||
For testing review_by_rounds, check the number of rounds which decides the rounds of the questionnaire that each assignment needs. | |||
*when used in round is nil | |||
<pre> | |||
it 'returns false' do | |||
expect(@review_by_rounds).to be nil | |||
end | |||
</pre> |
Latest revision as of 18:19, 18 December 2018
Introduction
Background
One of the fantastic features that Expertiza provides is for each assignment, it allows students providing reviews for others' jobs. This is done by offering a pre-defined questionnaire for the whole assignment, which is called the rubric for this assignment. Each rubric has several questions, each question provides a criterion to evaluate the performance from a specific angle. Then students can evaluate the entire job by rating and answering these questions. The team did that job can also get a detailed feedback for different aspects of the project.
Motivation
One of the problems for the review system is that for each assignment, different topics are included. Students can choose the one topic they like, and those topics fall into different categories. That is, an assignment always include different kind of projects. In Expertiza, the topic can be refactoring projects, testing projects, Mozilla projects, etc. Apparently, for different kinds of projects, we have different criteria to evaluate their performance. However, due to the current design of the system, all projects belong to the same assignment can only have the same rubric, which may lead to unnecessary criteria for some projects and inadequate criteria for other projects. To make sure all kinds of projects can be evaluated properly. We plan to refactor the system to allow assigning different rubies for different projects in a specified appointment.
Analysis & Plan
To realize such purpose, we have the following tasks need to be done:
Check Box
For current design, the rubric used for a specified assignment is set at the following page. To make sure users can choose to set different rubrics for different projects, we need first add a checkbox at this page. Therefore, users can use this box to indicate if they want to use the same rubric for all the projects or set some different rubrics for some projects.
Dropdown List
To make sure users can choose rubrics for each project, we should add a new dropdown list beside each project in the following page. The number of dropdown list should decide by review round number. Therefore, each time users choose to identify different rubrics for different projects, the list will appear and users can choose proper rubrics. The default rubric set at last page will be overwritten in this case.
Controller
Make changes to the controller so that these functions can work. Most of our logic control codes will be written in helpers/sign_up_sheet_helper.rb.
logic flowchart:
Database
Add questionnaire_id in sign_up_topics table to make sure the projects can always relate to correct rubrics
Test
Test what we changed to make sure it works as intended and not influence the original system.
Implement Steps
Task 1 Add checkbox
1. edit the file views/assignments/edit/_rubrics.html.erb
2. add a checkbox at the top of the page which says "Vary rubric by topic"
3. set the default state for this checkbox as unchecked
Task 2 Add dropdown list
1. edit the file views/assignment/edit/_general.html.erb
2. add a variable to indicate if the checkbox said in Task 1 is checked
3. create a new file under views/sign_up_sheet called "_rubrics_list.html.erb" to store the code for the dropdown list
4. edit the file views/sign_up_sheet/_table_line.html.erb
5. add the code to render the dropdown list
6. making a judgment before rendering the dropdown list, only if the checkbox in Task 1ist checked to render the page
Task 3 Change controllers
Change the update method in sign_up_sheet_controller.rb to make sure when new rubric is chosen for a specific topic, it can be stored in the database properly
Task 4 Create migration
1. create a new migration to add questionnaire_id in sign_up_topics
2. the questionnaire_id is a foreign key which references to questionnaires
3. change the file models/sign_up_topic.rb, and add the relations with the questionnaire, the relation between them is one-to-one
Task 5 Test the changes
1. make a test plan for the features that need to be tested
2. follow the test plan to write automatic test codes
3. evaluate the results and refactor codes
Test plan
We mainly use cucumber and capybara to test UI, and the following are detailed plans
Test checkbox
1. go to the rubrics page under the assignment
2. we can see the checkbox says "Vary rubric by topic" shows on this page
3. the initial state of the checkbox is unchecked
Test dropdown list
Scenario 1
1. the checkbox "Vary rubric by topic" is unchecked
2. go to the topics page under assignment
3.there are no dropdown lists beside topics
Scenario 2
1. the checkbox "Vary rubric by topic" is checked
2. go to the topics page under assignment
3. the table to show the topics have a row called "rubrics"
4. under this title, each topic has a dropdown list
Result
Test
Instead of testing UI, we did some unit tests using Rspec.
For testing update_topic_questionnaires.
- when attributes are nil
it 'returns false' do expect(assignment_form.update_topic_questionnaires(nil)).to be false end
- when attributes are not nil and at least one topic_questionnaire's id is nil or blank
let(:attributes) { [topic_questionnaire, topic_questionnaire2] } before(:each) do allow(topic_questionnaire).to receive(:[]).with(:id).and_return(nil) allow(topic_questionnaire2).to receive(:[]).with(:id).and_return(1) allow(TopicQuestionnaire).to receive(:where).with(sign_up_topic_id: SignUpTopic.where(assignment_id: 1)).and_return([]) allow(TopicQuestionnaire).to receive(:new).with(topic_questionnaire).and_return(topic_questionnaire) allow(TopicQuestionnaire).to receive(:find).with(1).and_return(topic_questionnaire2) end
- when both save and update_attributes method do not work
it 'changes @has_errors value to true and returns attributes (args)' do allow(topic_questionnaire).to receive(:save).and_return(false) allow(topic_questionnaire2).to receive(:update_attributes).with(topic_questionnaire2).and_return(false) expect(assignment_form.update_topic_questionnaires(attributes)).to eq(attributes) expect(assignment_form.instance_variable_get(:@has_errors)).to be true end
- when both save and update_attributes method work well
it 'returns attributes (args) and @has_errors value is nil' do allow(topic_questionnaire).to receive(:save).and_return(true) allow(topic_questionnaire2).to receive(:update_attributes).with(topic_questionnaire2).and_return(true) expect(assignment_form.update_topic_questionnaires(attributes)).to eq(attributes) expect(assignment_form.instance_variable_get(:@has_errors)).to be nil end
For testing review_by_rounds, check the number of rounds which decides the rounds of the questionnaire that each assignment needs.
- when used in round is nil
it 'returns false' do expect(@review_by_rounds).to be nil end