|
|
| (30 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| This wiki page describes the work and goals that have been completed and achieved while implementing Expertiza Project "''E1936. Specialized Rubrics for Different Topic Types (e.g., Servo project, refactoring project)''" for the CSC/ECE 517 "Object-Oriented Design and Development" class, in the Spring semester of the 2019.
| | Please see [http://wiki.expertiza.ncsu.edu/index.php/E1936_Specialized_Rubrics E1936_Specialized_Rubrics] |
| | |
| __TOC__
| |
| | |
| == Introduction ==
| |
| This Wiki page is created per CSC/ECE 517: "Object-Oriented Design and Development", Expertiza Project requirements (Spring, 2019).
| |
| | |
| Our [http://wiki.expertiza.ncsu.edu/index.php/E1936_Specialized_Rubrics#Team team] was assigned the project "[https://docs.google.com/document/d/1Ozw2Bj2u_LyeKUdDqAyQtz1stEIKrxkmRnwmNyUgmzw/edit#heading=h.z86zstcoj6u7 E1936. Specialized rubrics for different topic types (e.g., Servo project, refactoring project)]". This is an Expertiza project.
| |
| | |
| In this wiki page we describe the problem to be solved, how we solve that problem, and how we test our solution. We also provide helpful links and contact information for our team.
| |
| | |
| == Project Description ==
| |
| | |
| In this section, we discuss the problem we need to solve, the Expertiza feature this problem is related to, what needs to be done to solve the problem (at a high level), and the previous attempt to solve this problem.
| |
| | |
| === Problem Statement ===
| |
| | |
| This problem statement summarizes the problem as described in the project description "[https://docs.google.com/document/d/1Ozw2Bj2u_LyeKUdDqAyQtz1stEIKrxkmRnwmNyUgmzw/edit#heading=h.z86zstcoj6u7 E1936. Specialized rubrics for different topic types (e.g., Servo project, refactoring project)]".
| |
| | |
| In CSC/ECE 517, there are several types of topics that could be covered in a single class assignment.
| |
| * Code base
| |
| ** Expertiza
| |
| ** Mozilla
| |
| ** etc.
| |
| * Goal
| |
| ** Refactoring
| |
| ** Testing
| |
| ** etc.
| |
| However, currently we can only specify one kind of rubric for an assignment. This means that teams working on different topics will be evaluated using the same rubric. With this setup, it's not possible to fine-tune rubrics for different topics - rubrics tend to be overly general.
| |
| | |
| === Feature Description ===
| |
| | |
| The Expertiza project already has a feature that allows rubrics to vary by project phase or "round". The feature we will add in this project is a feature to allow rubrics to vary not only by round, but also by topic. This allows more flexibility to the instructor when setting up an assignment, so that they can use rubrics better suited to the tasks that students are performing in an assignment.
| |
| | |
| When this feature is complete, the following scenarios will be possible, and the instructor will be the judge of which scenario is the best fit to the assignment:
| |
| | |
| * One rubric for the assignment, period.
| |
| * A different rubric for each round, regardless of what topic different students are working on.
| |
| * A different rubric for each topic, regardless of the round.
| |
| * A different rubric for each phase of the assignment, and within a round, a different rubric for each topic.
| |
| | |
| === What Needs to be Done ===
| |
| | |
| This section summarizes the work as described in the project description "[https://docs.google.com/document/d/1Ozw2Bj2u_LyeKUdDqAyQtz1stEIKrxkmRnwmNyUgmzw/edit#heading=h.z86zstcoj6u7 E1936. Specialized rubrics for different topic types (e.g., Servo project, refactoring project)]".
| |
| | |
| * In assignment#edit page "Rubrics" tab, add a checkbox to specify whether rubric should vary by topic.
| |
| * In assignment#edit page "Topics" tab, add dropdown(s) next to each topic to specify which rubric(s) are associated with the topic.
| |
| ** The dropdown(s) should appear only if rubric should vary by topic, per the Rubrics tab.
| |
| ** The topic should have a multiple dropdowns (one for each round) only if rubrics vary by round, per the Rubrics tab.
| |
| ** By default, the rubric(s) for each course project will be the one(s) specified in “Rubrics” tab.
| |
| ** The dropdown value can overwrite the default rubric.
| |
| * Be careful when making changes to the code
| |
| ** The signup_sheet_controller should have as little to do as possible with the selection of rubrics.
| |
| ** Anything not germane to selecting topics should be in another controller or (more probably) a helper.
| |
| * Create a DB migration to allow rubric to vary by topic as well as by round.
| |
| | |
| ==== Screenshots ====
| |
| '''Current screenshot of Rubrics tab for an Assignment'''
| |
| (need to add a checkbox to indicate that rubric should vary by topic)
| |
| | |
| [[File:E1936 Screenshot Assignment Edit Rubrics.png]]
| |
| | |
| | |
| '''Current screenshot of Topics tab for an Assignment'''
| |
| (need to add a way to select rubrics for particular topics)
| |
| | |
| [[File:E1936 Screenshot Assignment Edit Topics.png]]
| |
| | |
| | |
| | |
| '''Proposed screenshot of Rubrics tab for an Assignment'''
| |
| | |
| [[File:E1936 Specialized Rubric rubric checkbox.png]]
| |
| | |
| | |
| '''Proposed screenshot of Topics tab for an Assignment'''
| |
| | |
| (drop down shown for only one topic here, but final product will have one for each topic)
| |
| | |
| [[File:E1936 Specialized Rubric dropdown menu.png]]
| |
| | |
| == Previous Work ==
| |
| | |
| There was previous attempt to implement this project in the Fall of 2018 semester by different team, but it was rejected due to several reasons. Our team was provided with all relevant information and materials about the design and implementation of the previously attempted project. We would like to acknowledge that we use previous implementation and design as reference and our starting point for our own implementation. The following is the list of links with the relevant materials that we have used for better project understanding, determining pros and cons of previously proposed changes, and establishing our design strategy and implementation:
| |
| | |
| # [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1873_Specialized_rubics_for_different_topic_types Wiki]
| |
| # [https://github.com/expertiza/expertiza/pull/1327 Pull Request]
| |
| # [https://drive.google.com/file/d/1UrLb0Ula_Lb_F4MOrdC-s6xng6B08VYu/view Video Demo]
| |
| | |
| After reviewing all relevant materials above, we concluded that the project implementation was incomplete and proposed design had significant flaw.
| |
| We identified main drawbacks and flaw as follows:
| |
| | |
| * Establishing incorrect ''many-to-one'' relationship between ''sign_up_topics'' and ''questionnaires'' tables:
| |
| ** Creating a DB migration to add ''questionnaire_id'' in the ''sign_up_topics'' table to store the dropdown data presumes ''one-to-one'' or ''many-to-one'' relationship between ''sign_up_topics'' and ''questionnaires'' tables
| |
| ** Correct relationship between ''sign_up_topics'' and ''questionnaires'' tables is ''many-to-many''. For example, each project topic may have various rubrics depending on the round it is used (e.g., for round 1, topic may have one rubric; but for round 2, the same topic may have different rubric)
| |
| ** Proposed previous migration changes would break DB design and would not allow to store correctly topic and its associated rubric per different round.
| |
| * Per project requirement specifications, "''The dropdown box should appear only if 'Vary rubric by topic' is selected in the Rubrics tab.''" This requirement was not met and ''Vary rubric by topic'' checkbox in the Rubrics tab was not implemented. Therefore, dropdown box with the list of rubrics appears regardless of whether "''Vary rubric by topic''" is selected or not.
| |
| * The dropdown box includes the list of all possible rubrics including default value. However, this condition can be improved. The dropdown box should list the rubrics that were created by the current user (professor who is logged in). This would allow to shorten the list of possible rubrics dramatically and help to avoid any misunderstanding and confusion while user selects desired rubric for the course project.
| |
| | |
| We found no further issues with the rest implementation and in other project materials. Wiki page is very useful for understanding design of proposed changes, finding drawbacks and the flaw. Considering all the pros and cons, and taking into account drawbacks of the proposed changes, our team evolved different approach that should successfully solve the problem stated for the project without without creating additional issues. Please see our [http://wiki.expertiza.ncsu.edu/index.php?title=E1936_Specialized_Rubrics#Project_Implementation Project Implementation] for more details.
| |
| | |
| == Previous Project Requirements ==
| |
| | |
| The project requirements originally specified how to alter the databases to support varying rubrics by topic.
| |
| | |
| ''Create a new table named topic_questionnaires with field:''
| |
| * ''id''
| |
| * ''topic_id''
| |
| * ''questionnaire_id''
| |
| * ''used_in_round''
| |
| | |
| However, the E1936 team successfully petitioned for this requirement to change to give us more freedom in the implementation. The original suggestion had a flaw. There is already an existing table in the Expertiza project called ''assignment_questionnaires'', which has columns including ''assignment_id'', ''questionnaire_id'', and ''used_in_round''. It is this table that maintains the knowledge of which rubric is used in which round of an assignment. If we were to add a new table to maintains the knowledge of which rubric is used in which topic of an assignment as originally suggested, then we would have a big problem. Knowledge of round-rubric relationship is duplicated. This would lead (best case scenario) to extra work, and (worst case scenario) to lots of bugs caused by inconsistencies in how these tables are handled with CRUD operations throughout the code. In other words, this is not a DRY solution.
| |
| | |
| == Project Implementation ==
| |
| | |
| In this section, we propose our solution to the problem, discuss implementation techniques and details, and list the goals we achieved.
| |
| | |
| We start by analyzing Entity Relationship Diagram where we state the solution to the problem. Further we discuss design strategy by breaking project problem into multiple staged sub-problems and provide solution for each of these sub-problems. We also discuss some implementation details and provide the list of involved files. Finally, we demonstrate how implemented feature can be used by providing ''How To'' sub-section.
| |
| | |
| === Entity Relationship Diagram ===
| |
| | |
| Our solution is best introduced by first discussing how the database will be modified.
| |
| | |
| [[File: E1936 ERD.png]]
| |
| | |
| We would like to propose our solution to the problem by representing Entity Relationship Diagram and discussing how relationship between ''sign_up_topics'' and ''questionnaires'' tables can be established to associate project's topic with a rubric.
| |
| | |
| ''Note'' that in the existing implementation neither ''sign_up_topics'' nor ''questionnaires'' tables have relation with each other. Meaning ''sign_up_topics'' table has no foreign key that points to the ''questionnaires'' table, and equivalently for the ''questionnaires'' table, it has no foreign key that points to the ''sign_up_topics'' table. This problem would easily be solved if the ''sign_up_topics'' and ''questionnaires'' tables have ''many-to-one'' relationship (which seems to be an appropriate approach since each topic has exactly one rubric). In this case we would create a DB migration to add ''questionnaire_id'' in the ''sign_up_topics''. However, this solution is incorrect as we stated in the [http://wiki.expertiza.ncsu.edu/index.php?title=E1936_Specialized_Rubrics#Previous_Work Previous Work] section.
| |
| | |
| The problem is that each topic has exactly one rubric per specific round. The same topic may have different rubric with various rounds. Therefore, ''many-to-one'' relationship would break DB design and would limit to have only one rubric per topic for each round. Hence, the correct is ''many-to-many'' relationship between the tables.
| |
| | |
| To solve this dilemma and properly establish ''many-to-many'' relationship between ''sign_up_topics'' and ''questionnaires'' tables, we came to the conclusion and can state solution to the core problem of the project:
| |
| * Create new ''topic_questionnaire'' table with foreign keys to each of the ''sign_up_topics'' and ''questionnaires'' tables.
| |
| | |
| By adding this new table, we create a many-to-many relationship between ''sign_up_topics'' and ''questionnaires'' tables and this would allow project topic to record multiple rubrics per each round. In the following Entity Relationship Diagram, we show existing DB design (in black) and proposed DB design changes (depicted in blue):
| |
| | |
| [[File: CSC520_Project_4_Diagram.PNG]]
| |
| | |
| Above Entity Relationship Diagram can be interpreted as follows:
| |
| * ''sign_up_topics'' table is connected with ''topic_questionnaire'' table through ''many-to-many'' relationship (i.e., project's topic may have multiple topic questionnaires per each round)
| |
| * ''topic_questionnaire'' table is connected with ''questionnaires'' tables via ''many-to-one'' relationship (i.e., topic questionnaire belongs to exactly one questionnaire)
| |
| | |
| To illustrate this we can use the following table that reflects new ''topic_questionnaire'' table at any point in time:
| |
| <table>
| |
| <tr>
| |
| <th>id</th>
| |
| <th>topic_id</th>
| |
| <th>questionnaire_id</th>
| |
| <th>used_in_round</th>
| |
| </tr>
| |
| <tr>
| |
| <td>1</td>
| |
| <td>1</td>
| |
| <td>1</td>
| |
| <td>1</td>
| |
| </tr>
| |
| <tr>
| |
| <td>2</td>
| |
| <td>1</td>
| |
| <td>2</td>
| |
| <td>2</td>
| |
| </tr>
| |
| <tr>
| |
| <td>3</td>
| |
| <td>1</td>
| |
| <td>3</td>
| |
| <td>3</td>
| |
| </tr>
| |
| </table>
| |
| | |
| Above table demonstrates that the same topic with id=1 may have various questionnaires (id's=1, 2, 3) depending on the round (round 1, 2, 3 and etc).
| |
| | |
| === Design Strategy ===
| |
| | |
| As we have concrete understanding of the problem and proposed solution, we would like break this problem into multiple more manageable sub-problems with specific tasks and well-defined solutions to these tasks. Generally, each sub-problem requires modification of one or more files. We would like to propose our Design Strategy as follows:
| |
| | |
| * Rubrics tab: create "''Vary rubric by topic''" checkbox
| |
| ** File: ''app/views/assignments/edit/_rubrics.html.erb''
| |
| ** PROBLEM: There is no way for the user to indicate that rubrics should vary by topic
| |
| ** SOLUTION: Add a new checkbox in the rubric tab to indicate whether rubrics vary by topic or not
| |
| | |
| * Topics tab: Preparation
| |
| ** File: ''app/views/assignments/edit.html.erb''
| |
| ** PROBLEM: We need to add/change content to the Topics tab, but unlike other tabs in the views, it does not have its own partial view to work with
| |
| ** SOLUTION: Refactor and move the code dealing with topics into separate ''app/views/assignments/edit/_topics.html.erb'' file
| |
| | |
| * Topics tab: Add "''Questionnaire''" column and review dropdown content
| |
| ** File: ''app/views/assignments/edit/_topics.html.erb''
| |
| ** PROBLEM: There is no way for the user to select rubric(s) for a particular topic
| |
| ** SOLUTION: Add a new column in the topics table for choosing rubric(s)
| |
| *** New column only visible if rubrics vary by topic, per Rubrics tab
| |
| *** New column has multiple rounds per topic only if rubrics vary by round, per Rubrics tab
| |
| *** Default value for any rubric is that rubric which is shown on Rubrics tab
| |
| *** Rubric(s) in the dropdown list is/are only those that were created by the currently logged in instructor
| |
| ** PROBLEM: The code in ''app/views/assignments/edit/_rubrics.html.erb'' includes dropdowns like the ones we need to add for topics, but that code is way too complex for its functionality (too heavy for a view, uses JavaScript)
| |
| ** SOLUTION: Use the code in ''app/views/assignments/edit/_rubrics.html.erb'' for inspiration only
| |
| *** Refactor and move as much actual work as possible out into helper methods that are simply called from this view
| |
| *** Views are not intended to do heavy lifting
| |
| *** Use as little JavaScript as possible, since this is a Ruby On Rails project, not a JS project
| |
| | |
| * <s>Create relationship between topics and rubrics
| |
| ** File: ''db/migrate''
| |
| ** PROBLEM: Questionnaires (of which rubrics are one type) are not linked to assignment topics
| |
| ** SOLUTION: Create a new migration to link rubrics to topics
| |
| *** Create a new table named topic_questionnaire with the following fields:
| |
| **** ''id''
| |
| **** ''topic_id''
| |
| **** ''questionnaire_id''
| |
| **** ''used_in_round''
| |
| *** This new table links ''sign_up_topics'' and ''questionnaires'' table through ''topic_questionnaire''</s>
| |
| | |
| * <s>Support new topic_questionnaire table with a model
| |
| ** File: ''/app/models/topic_questionnaire.rb''
| |
| ** PROBLEM: New table will not have any real support in the code after migration alone
| |
| ** SOLUTION: Create a model for the new table
| |
| *** Implement validations in the model file
| |
| *** Implement any needed methods in the model file</s>
| |
| | |
| * Add topic_id field to existing assignment_questionnaires table
| |
| ** File: ''db/migrate''
| |
| ** PROBLEM: We need to link Questionnaires (of which rubrics are one type) to assignment topics. We also need the ability to locate a questionnaire given any combination of the following: assignment, topic, round.
| |
| ** SOLUTION: Create a new migration which adds a topic_id field (which will reference the id field in the sign_up_topics table) to the existing assignment_questionnaires table.
| |
| | |
| * React to change in "''Vary rubric by topic''" checkbox
| |
| ** File: ''/app/models/topic_questionnaire.rb'' and ''app/helpers/assignment_helper.rb''
| |
| ** PROBLEM: A checkbox without any action taken when it is checked / unchecked is not useful
| |
| ** SOLUTION: Create method(s) in the topic_questionnaire model to react to this checkbox
| |
| *** Call new method(s) from the view ''app/views/assignments/edit/_topics.html.erb''
| |
| **** ''app/views/assignments/edit/_rubrics.html.erb'' reacts to the ''Review rubric varies by round?'' checkbox and so has code that is similar to what we'll need
| |
| **** As noted previously, we will use the code in ''app/views/assignments/edit/_rubrics.html.erb'' for inspiration only
| |
| *** When the user clicks 'Save' with the checkbox selected:
| |
| **** [TBD]
| |
| *** When the user clicks 'Save' with the checkbox un-selected:
| |
| **** [TBD]
| |
| *** Q & A
| |
| **** Q: What will happen if an assignment's rubrics vary by topic, then later this is changed so that the rubrics do not vary by topic?
| |
| **** A: [TBD]
| |
| **** Q: What will happen if an assignment's rubrics vary by round, then later this is changed so that the rubrics do not vary by round?
| |
| **** A: [TBD]
| |
| | |
| * Find questionnaire for a given assignment/round/topic
| |
| ** File: ''app/helpers/assignment_helper.rb''
| |
| ** PROBLEM: The ''assignment_questionnaire'' method finds a questionnaire for a given assignment/round, and would be a great place to include code to find a questionnaire for a given assignment/round/topic, however the method is not easy to work with
| |
| ** SOLUTION: Improve readability of the ''assignment_questionnaire'' method
| |
| *** Add comment block at the top describing the purpose and basic strategy of the method
| |
| *** Rename 'number' parameter to clarify the purpose of the parameter
| |
| ** SOLUTION: Expand functionality of the ''assignment_questionnaire'' method
| |
| *** Add ability to get assignment questionnaires by topic as well as by round
| |
| *** Like round, topic parameter should be optional
| |
| | |
| * Check if given assignment has rubric that varies by topic
| |
| ** File: ''app/models/assignment.rb''
| |
| ** PROBLEM: The assignment model has a method to check if rubric varies by round, but no such method for topic
| |
| ** SOLUTION: Add a new method ''varying_rubrics_by_topic?'' based on the existing method ''varying_rubrics_by_round?''
| |
| | |
| * Test all of the affected helpers
| |
| ** File: ''spec/helpers''
| |
| ** PROBLEM: We are going to add code to the file ''app/helpers/assignment_helper.rb'', but this helper does not have an RSpec test suite yet
| |
| ** SOLUTION: Add ''spec/helpers/assignment_helper_spec.rb'' file
| |
| *** Thoroughly test ''assignment_questionnaire'' method in the ''app/helpers/assignment_helper.rb''
| |
| ** PROBLEM: We may also need to create other helpers or helper methods as we get into the code
| |
| ** SOLUTION: Add tests for any newly created helpers or helper methods
| |
| | |
| * Test all of the affected models
| |
| ** File: ''spec/models''
| |
| ** PROBLEM: We are going to add code to the file ''app/models/assignment.rb'', but this model does not have an RSpec test suite yet
| |
| ** SOLUTION: Add ''spec/models/assignment_spec.rb'' file
| |
| *** Thoroughly test ''varying_rubrics_by_topic?'' method in ''app/helpers/assignment_spec.rb''
| |
| ** PROBLEM: We are going to add a new model ''/app/models/topic_questionnaire.rb''', which will need an RSpec test suite as well
| |
| ** SOLUTION: Add ''spec/models/topic_questionnaire_spec.rb'' file
| |
| *** Thoroughly test all of the new model's methods
| |
| | |
| === Implementation ===
| |
| | |
| In this section, we provide selected portions of modified code to illustrate how we implemented the new feature.
| |
| | |
| [to be populated after coding is complete]
| |
| | |
| === Files Involved ===
| |
| | |
| In this section, we list all files that were modified while implementing the new feature.
| |
| | |
| * [to be populated after coding is complete]
| |
| | |
| === How To ===
| |
| | |
| In this section, we describe how to use the newly implemented feature.
| |
| | |
| ==== Add a New Rubric ====
| |
| | |
| * [to be populated after coding is complete]
| |
| | |
| ==== Select a Rubric for a Topic ====
| |
| | |
| * [to be populated after coding is complete]
| |
| | |
| == Project Testing ==
| |
| To test code implementation, correctness of the added feature, verify that the team did not break any existing functionalities in the Expertiza, and ensure that no bugs were introduced in the code, we utilized the following Test Strategy:
| |
| # Code inspection
| |
| # Run and pass existing RSpec Tests
| |
| # Develop New RSpec Tests
| |
| # Run through live UI to test a feature using Expertiza instance
| |
| | |
| === Rspec Testing ===
| |
| | |
| Our strategy for gaining confidence that our code changes did not break anything was as follows:
| |
| * Run all existing RSpec tests on any changed files, after our changes, to ensure that we have not introduced any failures.
| |
| * The commands and results are shown below.
| |
| | |
| <pre>
| |
| [to be populated after coding is complete]
| |
| </pre>
| |
| | |
| * Write new comprehensive RSpec tests, for all new methods.
| |
| * Run these tests, to ensure that the new code works as intended.
| |
| * The commands and results are shown below.
| |
| | |
| <pre>
| |
| [to be populated after coding is complete]
| |
| </pre>
| |
| | |
| * The test suite for a single new method is below.
| |
| * There are many such suites in (TODO add filename(s)).
| |
| * This example illustrates our general strategy:
| |
| ** test missing input
| |
| ** test bad input
| |
| ** test various acceptable forms of input
| |
| ** test scenarios that lead to "true" and to "false" return values
| |
| | |
| <pre>
| |
| [to be populated after coding is complete]
| |
| </pre>
| |
| | |
| === UI Testing ===
| |
| Here we describe UI Testing that has performed. Include Screenshots maybe?
| |
| | |
| == Conclusion ==
| |
| | |
| [to be populated after coding is complete]
| |
| | |
| == Useful Links ==
| |
| In this section we provide useful links related to the work that has been completed by our team.
| |
| # [https://github.com/gabalmat/expertiza Forked Project Repository]
| |
| # [(add link here) Pull Request]
| |
| # [(add link here) Video Demonstration of Specialized Rubrics]
| |
| # [(add link here) Video Demonstration of Passing Rspec Tests]
| |
| | |
| == Team ==
| |
| === Contacts ===
| |
| We have 4 (four) members in our team and 1 (one) mentor was also assigned to us. For any project-related questions/concerns please contact us:
| |
| * Aurora Tiffany-Davis - attiffan@ncsu.edu
| |
| * Ginger Balmat - gabalmat@ncsu.edu
| |
| * Joe Hutchinson - jehutch3@ncsu.edu
| |
| * Nikolay Titov - ngtitov@ncsu.edu
| |
| * Zhewei Hu (mentor) - zhu6@ncsu.edu
| |
| | |
| === Contribution Based Inquiry ===
| |
| For any questions or concerns related to the project design, implementation, testing of the feature developed by our team, please contact an engineer who worked closely on the topic of your interest based on the [https://docs.google.com/spreadsheets/d/11uxuI_5gaC-FQDwIOxtCaN3HZ9bfWDJgIjFemtYV5jE/edit?ts=5ca222f3#gid=0 Contribution Assignment Grid] we created to keep track of our progress and contribution. Please carbon copy all team members and our mentor in the email.
| |
| | |
| == References ==
| |
| * Expertiza Web [https://github.com/expertiza/expertiza#expertiza]
| |
| * Expertiza on GitHub [https://github.com/expertiza/expertiza]
| |
| * Expertiza Wiki [http://wikis.lib.ncsu.edu/index.php/Expertiza]
| |
| * Rspec Documentation [https://relishapp.com/rspec]
| |
| * Wiki with Previous Work [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2018/E1873_Specialized_rubics_for_different_topic_types]
| |
| * Pull Request of the Previous Work [https://github.com/expertiza/expertiza/pull/1327]
| |