CSC/ECE 517 Spring 2013/OSS E734: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
(Created page with "[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.] This page is the design documentation for Expertiza pr...")
 
Line 15: Line 15:




= Expertiza Project E734: Analytics Proposal =
= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.


The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed. Thus, by mining such information, Expertiza can be improved by leveraging the results that are obtained.  
The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.  


A previous Expertiza project (E732) has added a new module that helps the instructor design courses on Expertiza that will facilitate building a better learning environment. On the base of E732, our project aims at helping assignment authors and reviewers make a better use of the reviewing process and get a more comprehensive feedback from the review information.
This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.  


== Objective ==
== Objective ==
To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on. The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected to make a comparison for each review or meta review.
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.  
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.


To start with, our application will answer the following questions:
== Requirements ==  
 
In order for us to come up with a viable and well designed framework, first we came up with a list of requirement that the frame work have to meet using the Object Oriented Design Principles and Design Patterns we learn in the class.
a. What information can the instructor and the assignment author gain from scores and text for different questions of a particular review/meta review?
*...(TODO)
 
*A clear separation front end (displaying the data) and back end (gathering the data)
b. What information can the instructor and the assignment author gain from all the scores and text for the same question of all the related reviews?
 
c. What information can the instructor and the assignment author gain from the changes of scores regarding the same review question in review, re-review, and meta review?
 
d. Who has the access to the above information?
 
== Project Description ==
Each of the above mentioned questions can be handled as follows:
 
* a. What information can the instructor and the assignment author gain from scores and text for different questions of a particular review/meta review?
 
Rubrics are essentially a predetermined set of questions provided to every reviewer in order for them to be able to provide a feedback on the work they are assessing.  For a particular question in a review, we would be therefore taking into account the following two parameters namely score assigned by the reviewer for question of a review, and the text associated with each score.  Analyzing the above mentioned parameters, the instructor and the assignment author would be able to come up with an overall idea on how the different aspect of the assignment is evaluated and what improvement is needed.
 
* b. What information can the instructor and the assignment author gain from all the scores and text for the same question of all the related reviews?
 
A particular question in a review may be related to one aspect of the evaluation of the assignment. Since a single feedback could be parochial and subjective, collecting and analyzing all the feedbacks from all the reviews could help the instructor and the assignment author get a better understanding about how the work is evaluated. By calculating the average score of a particular question, we will also give weight to all the feedbacks. By comparing each score to the average score, the feedbacks with scores closer to the average are considered more impartial and the text from these feedbacks should be more valuable.
 
* c. What information can the instructor and the assignment author gain from the changes of scores regarding the same review question in review, re-review, and meta review?
 
With each review performed, the assignment author will gain feedbacks from reviews and come up with some improvements according to the scores and comments. These improvements can be presented in the resubmission so that the re-review should reflect how these improvement is evaluated. Thus analyzing the trend in scores of different reviews (review, re-review and meta review) we would be able to come up with a general idea on how well the author understands former reviews and gets useful suggestions from former reviews.
 
* d. Who has the access to the above information?
 
Keep the above information only accessible for the instructor and the assignment author. Try to make this interface as flexible as possible while keeping in mind security vulnerabilities such as SQL injection attacks.
 
== Requirements ==
*Instructors or Teaching assistants must be able to see a comparison among different aspects in a review.
*Instructors or Teaching assistants must be able to see a comparison among different feedbacks towards the same question in all reviews of an assignments.
*Instructors or Teaching assistants must be able to see a comparison among different feedbacks towards the same question in all reviews of an assignments.
*Instructors or Teaching assistants must be able to see the trend of the score changes with respect to time, which is from review to re-review and finally to meta review.
*Instructors or Teaching assistants must be able to see the trend of the score changes with respect to time, which is from review to re-review and finally to meta review.
*All the above information should be kept to the instructor, TAs and the respecting author.
*All the above information should be kept to the instructor, TAs and the respecting author.

Revision as of 19:24, 28 April 2013

Write-up of This Topic.

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main page: http://wikis.lib.ncsu.edu/index.php/Expertiza


Team Members

Hao Liu (hliu11@ncsu.edu)

Zhuowei Wang (zwang18@ncsu.edu)

Chunxue Yang (cyang14@ncsu.edu)


Introduction

The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

Objective

The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features. Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

Requirements

In order for us to come up with a viable and well designed framework, first we came up with a list of requirement that the frame work have to meet using the Object Oriented Design Principles and Design Patterns we learn in the class.

  • ...(TODO)
  • A clear separation front end (displaying the data) and back end (gathering the data)
  • Instructors or Teaching assistants must be able to see a comparison among different feedbacks towards the same question in all reviews of an assignments.
  • Instructors or Teaching assistants must be able to see the trend of the score changes with respect to time, which is from review to re-review and finally to meta review.
  • All the above information should be kept to the instructor, TAs and the respecting author.
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2013/OSS_E734&oldid=75214"