CSC/ECE 517 Spring 2024 - E2414 Grading Audit Trail
Problem Statement
NC State faculty members and students use Expertiza Open-Source Website to submit assignments, grade them, form teams, and among other features. In this project (E2414), we address the issue of instructors being able to freely edit a grade with no tracking of what changes are made by whom. There is a need for a way to view all of the changes made to a grade in order to provide accountability
Program Objectives
Grading Audit Trail must be implemented with the following information to be stored:
1. When an instructor assigns a grade, there needs to be an indication of who did it and when it was done.
2. Comments previously provided by other instructors must also be preserved.
This information needs to be stored every time an instructor edits a grade / comments and clicks the save button. The grading audit trail can probably pattern off the submission records history on Expertiza. The submission record page can be reached by logging in as instructor -> Manage -> Assignments -> View Submissions -> History. At the minimum, a grading log entry must include the instructor id, assignment id, student id, grade comment and timestamp.
Previous work from other teams
Project E2383 tackled the same issue of creating a grading audit trail. Our work was based off of this implementation, which provided much of the core functionality of creating a history record for each grade change, as well as creating a page for displaying grading history. However, there were some shortcomings of this implementation which we attempt to address in our implementation
Issues:
1. Poor link placement and naming
2. Minimal testing of controller for grading history
3. Grading history table used equal-width columns which caused long comments to have large vertical span
4. Several instances of controller code that could be refactored into model
Program 4 Design Document
Task Overview
In program 3, we implemented functionality to maintain an audit trail of all changes made to grades by instructors
The work to be completed for program 4 consists of 2 major parts: Refactoring changes from program 3 and Implementing logging support for grade history
Refactoring Changes from Program 3
- The table on the grading history page is too large and there is large amounts of whitespace between columns
- The method assignment_for_history in grading_history.rb is an unnecessary class method, and should be refactored as an instance method utilizing polymorphism
- Creation of grading history records should be done using filters, adhering to single responsibility principle in non-grading-related controllers
Logging Support for Grade History
- Implement logging using ElasticSearch Logstash Kibana (ELK), which is currently in use for logging-related tasks in Expertiza
- Logging should be done for all grade changes made in Expertiza, including both for assignments and reviews
UML Diagram
Design Patterns
We will apply several design patterns and principles in our implementation to allow us to write simpler and more maintainable code.
Many of our refactoring changes from program 3 apply the single responsibility principle. By refactoring the class method assignment_for_history into a new class, we decouple the GradingHistory class from Assignments which removes an unnecessary responsibility. Additionally, implementing creation of grading history records as a filter extracts this code from controller methods and into its own function, creating a better separation of responsibilities.
Our implementation of the grading history functionality can use the Memento design pattern. Our implementation keeps track of the state of a grade for an assignment or review anytime the grade is changed. Storing the state after every change provides the ability to identify potential malicious changes to a grade and revert the grade back to what it previously was. Applying the memento design pattern can make our code more readable and easier to understand.
Lastly, our implementation for logging functionality can follow the Mediator design pattern. The ExpertizaLogger and LoggerMessage classes provide a consistent interface for creating log messages, regardless of what application component is creating the log message. By utilizing these classes in our implementation, we avoid the need to directly interact with the logging system, instead relying on the abstraction of a log message. Using this pattern can greatly simplify our implementation.
Plan of Work
Grading History Table Changes
Problem
The above image shows the current layout of the Grading History page. We can see that the table takes up the entire width of the screen, and there is a large amount of unnecessary whitespace in the comments section when the comment is relatively short. Adjusting the column widths would provide a more compact view that is easier for users to read
Solution
We will make the following changes to app/views/grading_histories/index.html.erb to make the table more readable:
1. Resize existing columns with fixed widths to minimize whitespace
2. Dynamically size the 'Comments' column to reduce unnecessary whitespace, up to a fixed maximum width
3. The resulting table will be centered in the middle of the screen
Refactor class method assignment_for_history in grading_history.rb
Problem:
Solution
Refactor Grading History Creation using filters
Problem: The project focused on improving the grading audit trail for the Expertiza Website. The problem faced was the need for a method that could reliably identify the relevant assignment context, whether a submission or a review, so as to obtain the grading history accurately. The previous implementation may have been not sufficiently encapsulated, leading to potential issues when trying to extend functionality or adapt to new requirements.
Solution:
- Dynamic Context Identification: Implemented a method that determines the assignment context based on the grading type :'Submission' or 'Review'.
- For 'Submission': The method uses the associated AssignmentTeam to find the related assignment.
- For 'Review': The method identifies the AssignmentParticipant first to navigate to the corresponding assignment.
- Adherence to Single Responsibility Principle: The refactored method focuses on a single task.
- Encapsulation of Query Logic: Introduced scopes or helper methods to manage complex ActiveRecord queries, keeping the controller code concise.
- Improved Maintainability and Clarity: The refactoring effort led to clearer and more maintainable code, aligning with the project's goal of a trustworthy grading audit trail.
Logging Support for Grade History
Problem
Solution
Implementation
Our project's implementation hinged on a blend of time-tested techniques and innovative practices. We began by adopting Test-Driven Development (TDD), writing comprehensive test cases using the TA-provided skeletons as a framework. This ensured that every piece of functionality was verified. Pair programming sessions enhanced our collaborative efforts, enabling real-time code reviews and knowledge sharing, which fostered a high level of code quality and team cohesion. We also leveraged Continuous Integration (CI) and Continuous Deployment (CD) pipelines to streamline our development process. This allowed for automated testing and deployment of our code, ensuring that changes were tested and integrated frequently. The result was a reduction in integration problems and more adaptive and faster release cycles. Moreover, refactoring was an integral part of our workflow. It was not just a one-time event but a continual process of improving the code structure without altering its external behavior. This was crucial for keeping the codebase clean, efficient, and ready for future extensions or changes. By utilizing version control best practices, we ensured that our codebase remained manageable and navigable. Feature branching allowed us to work on different aspects of the project simultaneously without disrupting the main codebase, and pull requests facilitated peer reviews before code was merged.
Creating GradingHistoriesController
The grading_histories_controller.rb functionality created on Ruby on Rails application serves as a critical component for managing and displaying the grading history of reviews in an educational setting. Rails ensures the integration with the model and view layers, facilitating a robust and intuitive interface for users to interact with grading data. Through the implementation of callbacks for pre-action configurations and the inclusion of authorization logic via concerns, the application effectively enforces security and role-based access control. This ensures that only authorized users such as administrators, instructors, and teaching assistants can access sensitive grading information based on their association with specific courses or assignments. The controller is adept at handling various user permissions, distinguishing between submission and review grading histories, and organizing all related entries in a user friendly manner by displaying them in reverse chronological order. This organization enhances the application's usability by allowing users to easily navigate through the grading history.
Creating Model GradingHistory
The grading_history.rb model,has been created and enhanced with a refactoring that introduces a class method assignment_for_history. This method significantly extends the model's functionality by determining the context of grading—whether it pertains to a submission or a review based on the type of grading activity. For submissions, it identifies the associated AssignmentTeam using the graded_member_id to retrieve the corresponding assignment. In the case of reviews, it uses the participant_id to find the specific AssignmentParticipant, facilitating a direct link to the reviewed assignment or activity through the ReviewGrade model. This addition underscores the model's pivotal role in not only linking grading records to instructors and assignments via belongs_to associations but also in dynamically determining the relevant assignment context based on grading type. This enhancement bolsters the model's capacity to provide a comprehensive and nuanced tracking mechanism, offering deeper insights into the historical analysis of grading actions across the platform.
Creating Model ReviewGradingHistory
The review_grading_history.rb model, inheriting from the controller, is tailored for peer review grading events, featuring protection against mass-assignment vulnerabilities and establishing a relationship with the Participant class to track review grades within the system.
Creating Model SubmissionGradingHistory
The submission_grading_history.rb model extends the controller to focus on submission-related grading records, leveraging attr_protected to enhance security and associating with the Team class to directly relate grades to submitting teams within the platform.
Creating View grading_histories
The index.html view for grading_histories has been created to enhance performance and readability. It presents a grade history in a striped table format, centered around a dynamic header that adapts to display grading information pertinent to submissions or reviews. The created and refactored code uses conditional logic to efficiently handle potential nil records, ensuring that the display adapts gracefully in the absence of grading data. When available, the table populates with comprehensive details such as the instructor's full name, grade, comments, and the precise timestamp of grading—presented in an orderly fashion.
Updating Grades Controller
We introduced important functionalities to improve grading transparency and history tracking on the grades_controller.rb. An addition to the save_grade_and_comment_for_submission method is the creation of a GradingHistory record each time a submission is graded. This record stores vital information, including the instructor's ID, assignment ID, the type of graded item ('Submission'), the graded team's ID, along with the given grade and accompanying comment. This development is a significant stride in grading auditability, as it ensures every grading action is logged and traceable within the system. The method also encapsulates error handling to alert users in the event of a failure in saving the grade and comment, thereby maintaining a user-friendly interface.
Updating Review Mapping Controller
We integrated a robust suite of methods to facilitate peer review assignments and management within our educational platform on the review_mapping_controller.rb. The introduction of dynamic reviewer assignments allows for flexible peer evaluation processes, aligning with the evolving needs of academic settings. The calibration feature provides a mechanism for instructors to gauge the quality of reviews, ensuring that the assessment process is both fair and consistent. One the most important improvements is the inclusion of grading history records for reviews. With this feature, the grading process for reviewers is not just a transitory evaluation but a documented history, contributing to transparent and traceable assessment practices. This aspect of the enhancement not only adds depth to the instructor's oversight capabilities but also enriches the feedback loop for students. The controller now supports more organized review policies, checking for outstanding reviews and allowing assignments to adhere strictly to the configured review limits. These checks prevent over-assignment of reviews, thereby maintaining a balanced workload among participants and ensuring the quality of peer assessments. The enhanced error handling and user feedback mechanisms contribute to a smoother user experience by providing clearer guidance and actionable insights during exception scenarios, such as attempting to delete a review mapping in progress or assigning a self-review.
Updating Review Grade Model
The review_grade.rb model was updated with the find_graded_member method contribute to deepening the contextual understanding of review grades within the system. This method introduces the ability to directly associate a graded review with its corresponding assignment through the participant's parent relationship. By encapsulating this logic within the model, the system design adheres more closely to the principles of object-oriented programming, promoting encapsulation and cohesion. This change has a more intuitive and maintainable approach to accessing assignment details from a review grade instance. The find_graded_member method abstracts the complexity of traversing associated records, thereby simplifying external calls and enhancing code readability.
Update Views Reports ReviewReport HTML
The new code added for the views/reports/_review_report.html.erb moves the location of the Grading History link. The link had previously been placed directly adjacent to the grade entry form, cluttering the view as this link unnecessarily extended the width of the column including the grade entry form. With this change we move the Grading History link under the grade entry form, making this column more compact and allowing additional space for other columns
Update Spec Factories rb file
The code we added in the spec/factories/factories.rb file introduces a new factory for GradingHistory, which is designed to create test data for grading history records within the application. This factory sets up a mock GradingHistory object with predefined attributes such as instructor_id, assignment_id, graded_item_type, graded_member_id, grade, and comment. This addition allows for the testing of features related to the tracking and displaying of grading histories in the application, facilitating the assessment of functionalities that involve historical grading data for assignments, submissions, or participants. The inclusion of this factory enhances the test suite by providing a means to easily generate and manipulate grading history data for various testing scenarios.
Testing Grading Audit Trail
We implemented the test skeletons provided by our TA to enhance our testing framework. By integrating these pre-constructed scenarios into our test suite, we have strengthened the reliability of our application. These skeletons served as a starting point, which we tailored to align with the unique requirements and functionalities of our system. Through a process of refactoring, we eliminated redundant tests, introduced additional scenarios to cover new functionalities, and modified existing tests to better reflect the behaviors and permissions associated with the GradingHistoriesController. This approach has not only expedited our testing phase but has also granted us deeper insights into the intricate permissions and workflows within our application. By verifying each role's access and the corresponding outcomes, we ensured that our application upheld its stringent security and functionality standards.
Test code for Grading History Controller
The grading_histories_controller_spec.rb spec tests various scenarios to ensure proper access control and functionality of the grading history feature within an educational app. It uses fabricated data for users and assignments to check permissions, verifying that only administrators, instructors, and TAs associated with an assignment can view its grading history. The tests confirm that unauthorized roles like students are denied access. Additionally, it tests that the controller sets up necessary variables correctly, depending on whether the context is a submission or review. These tests safeguard against unauthorized access and ensure the reliability of the grading history tracking system.
Updated views for Grading Audit Trail
Submission Grading History Views
The following views demonstrate UI for viewing submission history. The view below can be accessed through the following steps when logged in as an instructor
1. Click the "Manage..." button in the navigation bar
2. For the desired assignment, click the "View Submissions" icon from the list of actions
The view below displays a list of submissions for the given assignment. From this page, instructors are able to assign grades to submissions, and also can see the grading history for the submission by following the "Grading history" link.
After clicking on "Grading history", you are taken to the view below. The instructor is displayed with a listing of changes made to the grade, with the most recent changes appearing first in the list. A complete history of all grades and comments for this submission is displayed.
Review Grading History Views
Similar views exist for viewing the grading history of Reviews. The view below can be accessed through the following steps when logged in as an instructor
1. Click the "Manage..." button in the navigation bar
2. For the desired assignment, click the "View reports" icon from the list of actions
3. Select "Review report" from the dropdown and click View
From this view, an instructor can assign a grade for a review by another student. Below the grade assignment form, there is a link to view the grading history of the review.
Following the "Grading history" link brings the user to the view below. Similar to assignments this provides a complete listing of all changes made to the review grade in tabular format. The page has a header that dynamically changes to specify the graded item type and graded student/team.
In our project, we implemented an efficient and user-friendly interface for managing grading histories within the Expertiza, enhancing both the submission and review processes. The screenshots provided demonstrate a seamless integration of a "Grading History" feature, accessible via links next to each team and individual's submission record. This advancement not only streamlines the grading workflow but also fosters transparency, allowing instructors to track grading progress and view detailed feedback over time. This gives traceability of what TA graded what submission, hence meeting our project objectives. We focused on providing clear and concise information, with each page displaying a log of grades and comments, including timestamps for each entry.
Results
The implementation of the Grading Audit Trail in the Expertiza platform has significantly enhanced the accountability and transparency of the grading process. The project E2414 has successfully addressed the critical need for an auditable grading system. This new feature reliably records the identity of the instructor making the change, the time of the modification, and preserves the lineage of feedback provided by multiple evaluators. Throughout the project, our team learned the importance of robust record-keeping within educational software. The development emphasized the need for precision in logging activities to ensure that all stakeholders, including students, instructors, and administrative staff, can trust the grading system's fairness and accuracy. The project underscored the complexity of implementing new features in a live environment, especially regarding data integrity and user interface design. The Grading Audit Trail now captures and displays the instructor's identity and the exact timestamp when a grade is assigned or edited, fulfilling the requirement for detailed traceability. The system retains and displays all previous comments from different instructors, ensuring that the entire feedback history is available for review. These improvements meets the project's initial objectives but also add value to the educational experience on the Expertiza platform by bolstering the credibility of the grading process.
Contributors to this project
Javier Badillo (unityid: jebadill, github: jebadillo69) Tyler Enck (unityid: tsenck, github: TylerEnck) Biniyam Sima (unityid: btsima, github: biniyamsima) Github repo https://github.com/TylerEnck/expertiza Link to PR https://github.com/expertiza/expertiza/pull/2774