Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-05-04T18:15:59Z

Naherrma:

== Introduction and Problem Statement ==
Expertiza has the ability to require self-reviews on projects, however, these self-review scores do not have much functionality. In the current system, self-review scores are created and stored, but are not used afterwards. This project seeks to improve upon the self-review system, and extend its functionality. The main points of this project are as follows:

* Create a method to calculate a derived score between self and peer-reviews. The closer the self-review is to the peer-review, the greater the score. Display this score on the "View Scores" page.
* Allow instructors to decide if peer-reviews should be visible before self-review completion. Display the correct review scores accordingly.
* Display the self-review scores in the "View Scores" and heat map of peer-reviews, denoting that they are a different type of review.

== Implementation ==
; Allow instructors to decide if students should be allowed to view peer-reviewers before completing their self-review.

: Currently when instructors create a new assignment, or edit an existing one, they have the option to require self-reviews.

: [[File:Allow Self Review.PNG]]

: This will be updated to ask instructors if peer-reviews should be visible before self-reviews are completed.

: [[File:Allow Self Review Edited.PNG]]

: When checked the system will check if a student has completed a self-review when they access the scores view. If they have not completed their self-review they will see a message indicating such. If they have completed the review, they will see their peer-reviews as normal.

; Find or create a function that will derive a score from self-reviews and peer-reviews.

: This score should be reflective of the difference between the students self-review score, and their peer's reviews. The purpose behind this score is to teach students to become better at reviewing their own work. By providing a score that reflects how similar their reviews are to their peer's reviews, students receive feedback on their self-assessment abilities. More information on the function creation/decision proccess can be found in the section below.

; Create the different displays for self-reviews

: Instructors and students both have a heat-map view of scores from peer-reviews

: [[File:Instructor Review Heatmap.png]]

:(Instructor view pictured) student's self-review scores will be added to this view, but need to be denoted in a way that makes it easy to see this is a self-review not a peer-review. One potential option is to use an icon that shows this is a self-review score.

: [[File:HeatMapSelfie.png]]

: Which makes it simple for users to understand that particular column represents the scores from a self-review. Other potential options are to title the column "Self-review", or to highlight/border the column with a particular color (ex. blue), both denoting that the score comes from a self-review. Similar approaches can be taken to differentiate between self-review and peer-review scores in the regular "view scores" page.

: [[File:SelfHeatMapSelfie.png]]
: (Student view pictured) The students can see their self review alongside the peer reviews.

: [[File:LimitedSelfReview.png]]
: The instructor has the option as described previously to limit students access to peer reviews until after they have submitted a self review. Students will see "You have to submit self-review under 'Your work' before checking 'Your scores'." next to "Your scores" until they submit a self review.

== Score Formulas ==
There are many possible methods of deriving a score between self and peer-reviews. Each formula has the same goal, to derive a meaningful score between the peer and self-reviews. Thus, giving feedback to students on their self-reviewing, hopefully increasing their abilities in this area. Several potential approaches are available, each with different pros and cons. The simpler of approaches involves only the self and peer-review scores, and the more complex approaches may involve several other factors.

The simplest approach is to simply take the difference between the average peer-review score, and the self-review score. This rewards the students for good self-reviewing ability, and evenly punishes students for over and underrating themselves. A similar approach is utilized in several other systems, in which when a students self-review score is within 5% of their average peer-review score, they will receive the greater of the two grades. An extension of this can be made to allow a weight to be applied to the difference between the grades, and to allow instructors to manually enter the weight for each assignment.

[[File:Design Doc Naive Formula.PNG]]

A slight change to this approach is to take into account the peer score, so the greater your peers believe you performed, the greater your score is (proportional to the difference between reviews again). However, this, and similar approaches, should be avoided as they are potentially harmful. In the chart below, it can be seen that students with poor performance will be harshly punished for poor performance in their peer-reviews, their self and peer-review derived grade, and finally in their instructor grade. The self and peer-review derived grade should only be indicative of the students ability to self-review, not of the quality of the project.

[[File:Design Doc Take Score Into Account.PNG]]

Another approach is to derive the grade based on the percent difference between the self-review score, and the peer-review scores. This results in a smooth gradient which does not punish students as harshly as the previous two approaches. However, at low scores students will see greater swings in this derived grade.

[[File:Design Doc Pecentage Diff.PNG]]

These previous approaches involve only the peer and self-review scores, and are the simple approaches to the problem, the following options involve greater complexity, but seek to better solve the problem at hand.

The SPARK approach is discussed [https://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/343/258 here.] Which results in a score that should accurately reflect the relation between self and peer scores. The scores here will need to be adjusted to match the grading scheme of the project, a score of 1 is equivalent to a 100%, and the difference between 1 and the score is representative of a lower grade, regardless of the score being above or below 1. This method also includes a second score titled SAPA, Self-Assessment to Peer-Assessment factor. This score requires teams to evaluate each team member on the same self-review rubric. After this is completed team member's self-review scores are divided by their average teammate-review score to create the SAPA. A SAPA of > 1.0 indicates that the student rated themselves higher than their team mates did, a score of less than 1.0 indicates that students underrated themselves compared to their peers.

[[File:Design Doc Spark Formula.PNG]]

Two other approaches involving teammate participation are ranking and FCS(Fair Contribution Scoring).

In the ranking approach, students are asked to rank their team, including themselves, from 1 to n (where n is equal to team size), where 1 is the highest score, and n is the lowest score. Students are then assessed based on their average rank from teammates and the rank they awarded themselves. This approach is simple, but it forgoes the rubric approach that garners greater self-reflection in specific areas.

In the FCS approach, students are given a finite number of points to award to their teammates, including themselves. The more points that a person receives, the greater their perceived work contribution. At the end of the project students are assessed based on their average number of points received, and the points awarded to themselves. While the approach is very similar to the ranking approach there are important differences. With this approach students will need to weight the contributions of each member on a percentage of points basis, rather than on a, teammate A vs teammate B overall basis.

{| class="wikitable"
|+ Approach Pros and Cons
! Approach
! Pros
! Cons
|-
! Simple approaches
| Simple to implement, scoring is quick, direct feedback based only on student's self-review ability
| Does not take into account teammate assessments, has potential pitfalls at low peer-review scores
|-
! SPARK approach
| Simple to implement, includes SAPA which shows how self-reviews compare to teammate-reviews
| A separate metric must be calculated, stored, and displayed, team-reviews will slow the process
|-
! Ranking approach
| A very simple approach, can be implemented similar to the bidding page
| Loss of some self-reflection without a rubric
|-
! FCS approach
| A slightly improved ranking approach, scores may be more accurate than just ranking
| Loss of some self-reflection without a rubric
|}

The current implementation focuses on the simple approach of finding the difference between the two review scores and subtracting that from the maximum possible grade. For future work on this project it is recommended that teams combine multiple approaches to have the pros of several different score types. For example, one can combine the simple approaches, and the FCS approach by calculating both, and using a percentage of each one to create the final score. The prototype pattern can be used to simplify the creation of these hybridized approaches.

== Future Work ==
The work done in this project can be built upon by future groups. Several areas can be improved upon, or new functionality can be added.
* Refactoring of the ResponseMap features. This feature is currently rather complicated, accessing scores from reviews is difficult, and requires several method calls.
* Implementing multiple score approaches, or hybridizing the approach. A small addition to the instructor UI could allow them to select from a list of possible approaches, or multiple approaches to combine. Perhaps each selected approach can have a weight input from the UI too.
* Improving the response times of the grades page in instructor view.

== Testing ==
Test cases are as follows:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The derived scores can be saved to the database.
* Self-review scores are displayed with peer-review scores
* Student can only see their own self-review

== Files Changed ==
* app/views/assignments/edit/_review_strategy.html.erb
* db/migrate/20180429203828_add_columndisplaypeerreviewbeforeselfreviewcomplete.rb
* db/schema.rb
* app/models/self_review_response_map.rb
* app/views/grades/_view_heatgrid.html.erb
* app/views/grades/view_team.html.erb
* app/models/response.rb
* app/models/vm_question_response.rb

== Reference ==
[https://github.com/YourSignificantOtter/expertiza Our repo link]

[https://github.com/expertiza/expertiza/pull/1183 Our pull request link]

[https://youtu.be/-u5MF9G3ViY Our demonstration video]

Edward F. Gehringer, "Self-assessment to improve learning and evaluation," ASEE Annual Conference and Exposition, Columbus, OH, June 25–28, 2017

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-05-04T18:10:47Z

Naherrma: /* Score Formulas */

== Introduction and Problem Statement ==
Expertiza has the ability to require self-reviews on projects, however, these self-review scores do not have much functionality. In the current system, self-review scores are created and stored, but are not used afterwards. This project seeks to improve upon the self-review system, and extend its functionality. The main points of this project are as follows:

* Create a method to calculate a derived score between self and peer-reviews. The closer the self-review is to the peer-review, the greater the score. Display this score on the "View Scores" page.
* Allow instructors to decide if peer-reviews should be visible before self-review completion. Display the correct review scores accordingly.
* Display the self-review scores in the "View Scores" and heat map of peer-reviews, denoting that they are a different type of review.

== Implementation ==
; Allow instructors to decide if students should be allowed to view peer-reviewers before completing their self-review.

: Currently when instructors create a new assignment, or edit an existing one, they have the option to require self-reviews.

: [[File:Allow Self Review.PNG]]

: This will be updated to ask instructors if peer-reviews should be visible before self-reviews are completed.

: [[File:Allow Self Review Edited.PNG]]

: When checked the system will check if a student has completed a self-review when they access the scores view. If they have not completed their self-review they will see a message indicating such. If they have completed the review, they will see their peer-reviews as normal.

; Find or create a function that will derive a score from self-reviews and peer-reviews.

: This score should be reflective of the difference between the students self-review score, and their peer's reviews. The purpose behind this score is to teach students to become better at reviewing their own work. By providing a score that reflects how similar their reviews are to their peer's reviews, students receive feedback on their self-assessment abilities. More information on the function creation/decision proccess can be found in the section below.

; Create the different displays for self-reviews

: Instructors and students both have a heat-map view of scores from peer-reviews

: [[File:Instructor Review Heatmap.png]]

:(Instructor view pictured) student's self-review scores will be added to this view, but need to be denoted in a way that makes it easy to see this is a self-review not a peer-review. One potential option is to use an icon that shows this is a self-review score.

: [[File:HeatMapSelfie.png]]

: Which makes it simple for users to understand that particular column represents the scores from a self-review. Other potential options are to title the column "Self-review", or to highlight/border the column with a particular color (ex. blue), both denoting that the score comes from a self-review. Similar approaches can be taken to differentiate between self-review and peer-review scores in the regular "view scores" page.

: [[File:SelfHeatMapSelfie.png]]
: (Student view pictured) The students can see their self review alongside the peer reviews.

: [[File:LimitedSelfReview.png]]
: The instructor has the option as described previously to limit students access to peer reviews until after they have submitted a self review. Students will see "You have to submit self-review under 'Your work' before checking 'Your scores'." next to "Your scores" until they submit a self review.

== Score Formulas ==
There are many possible methods of deriving a score between self and peer-reviews. Each formula has the same goal, to derive a meaningful score between the peer and self-reviews. Thus, giving feedback to students on their self-reviewing, hopefully increasing their abilities in this area. Several potential approaches are available, each with different pros and cons. The simpler of approaches involves only the self and peer-review scores, and the more complex approaches may involve several other factors.

The simplest approach is to simply take the difference between the average peer-review score, and the self-review score. This rewards the students for good self-reviewing ability, and evenly punishes students for over and underrating themselves. A similar approach is utilized in several other systems, in which when a students self-review score is within 5% of their average peer-review score, they will receive the greater of the two grades. An extension of this can be made to allow a weight to be applied to the difference between the grades, and to allow instructors to manually enter the weight for each assignment.

[[File:Design Doc Naive Formula.PNG]]

A slight change to this approach is to take into account the peer score, so the greater your peers believe you performed, the greater your score is (proportional to the difference between reviews again). However, this, and similar approaches, should be avoided as they are potentially harmful. In the chart below, it can be seen that students with poor performance will be harshly punished for poor performance in their peer-reviews, their self and peer-review derived grade, and finally in their instructor grade. The self and peer-review derived grade should only be indicative of the students ability to self-review, not of the quality of the project.

[[File:Design Doc Take Score Into Account.PNG]]

Another approach is to derive the grade based on the percent difference between the self-review score, and the peer-review scores. This results in a smooth gradient which does not punish students as harshly as the previous two approaches. However, at low scores students will see greater swings in this derived grade.

[[File:Design Doc Pecentage Diff.PNG]]

These previous approaches involve only the peer and self-review scores, and are the simple approaches to the problem, the following options involve greater complexity, but seek to better solve the problem at hand.

The SPARK approach is discussed [https://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/343/258 here.] Which results in a score that should accurately reflect the relation between self and peer scores. The scores here will need to be adjusted to match the grading scheme of the project, a score of 1 is equivalent to a 100%, and the difference between 1 and the score is representative of a lower grade, regardless of the score being above or below 1. This method also includes a second score titled SAPA, Self-Assessment to Peer-Assessment factor. This score requires teams to evaluate each team member on the same self-review rubric. After this is completed team member's self-review scores are divided by their average teammate-review score to create the SAPA. A SAPA of > 1.0 indicates that the student rated themselves higher than their team mates did, a score of less than 1.0 indicates that students underrated themselves compared to their peers.

[[File:Design Doc Spark Formula.PNG]]

Two other approaches involving teammate participation are ranking and FCS(Fair Contribution Scoring).

In the ranking approach, students are asked to rank their team, including themselves, from 1 to n (where n is equal to team size), where 1 is the highest score, and n is the lowest score. Students are then assessed based on their average rank from teammates and the rank they awarded themselves. This approach is simple, but it forgoes the rubric approach that garners greater self-reflection in specific areas.

In the FCS approach, students are given a finite number of points to award to their teammates, including themselves. The more points that a person receives, the greater their perceived work contribution. At the end of the project students are assessed based on their average number of points received, and the points awarded to themselves. While the approach is very similar to the ranking approach there are important differences. With this approach students will need to weight the contributions of each member on a percentage of points basis, rather than on a, teammate A vs teammate B overall basis.

{| class="wikitable"
|+ Approach Pros and Cons
! Approach
! Pros
! Cons
|-
! Simple approaches
| Simple to implement, scoring is quick, direct feedback based only on student's self-review ability
| Does not take into account teammate assessments, has potential pitfalls at low peer-review scores
|-
! SPARK approach
| Simple to implement, includes SAPA which shows how self-reviews compare to teammate-reviews
| A separate metric must be calculated, stored, and displayed, team-reviews will slow the process
|-
! Ranking approach
| A very simple approach, can be implemented similar to the bidding page
| Loss of some self-reflection without a rubric
|-
! FCS approach
| A slightly improved ranking approach, scores may be more accurate than just ranking
| Loss of some self-reflection without a rubric
|}

The current implementation focuses on the simple approach of finding the difference between the two review scores and subtracting that from the maximum possible grade. For future work on this project it is recommended that teams combine multiple approaches to have the pros of several different score types. For example, one can combine the simple approaches, and the FCS approach by calculating both, and using a percentage of each one to create the final score. The prototype pattern can be used to simplify the creation of these hybridized approaches.

== Testing ==
Test cases are as follows:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The derived scores can be saved to the database.
* Self-review scores are displayed with peer-review scores
* Student can only see their own self-review

== Files Changed ==
* app/views/assignments/edit/_review_strategy.html.erb
* db/migrate/20180429203828_add_columndisplaypeerreviewbeforeselfreviewcomplete.rb
* db/schema.rb
* app/models/self_review_response_map.rb
* app/views/grades/_view_heatgrid.html.erb
* app/views/grades/view_team.html.erb
* app/models/response.rb
* app/models/vm_question_response.rb

== Reference ==
[https://github.com/YourSignificantOtter/expertiza Our repo link]

[https://github.com/expertiza/expertiza/pull/1183 Our pull request link]

[https://youtu.be/-u5MF9G3ViY Our demonstration video]

Edward F. Gehringer, "Self-assessment to improve learning and evaluation," ASEE Annual Conference and Exposition, Columbus, OH, June 25–28, 2017

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T03:31:04Z

Naherrma: /* Score Formulas */

== Introduction and Problem Statement ==
Expertiza has the ability to require self-reviews on projects, however, these self-review scores do not have much functionality. In the current system, self-review scores are created and stored, but are not used afterwards. This project seeks to improve upon the self-review system, and extend its functionality. The main points of this project are as follows:

* Create a method to calculate a derived score between self and peer-reviews. The closer the self-review is to the peer-review, the greater the score. Display this score on the "View Scores" page.
* Allow instructors to decide if peer-reviews should be visible before self-review completion. Display the correct review scores accordingly.
* Display the self-review scores in the "View Scores" and heat map of peer-reviews, denoting that they are a different type of review.

== Planned Approach ==
; Allow instructors to decide if students should be allowed to view peer-reviewers before completing their self-review.

: Currently when instructors create a new assignment, or edit an existing one, they have the option to require self-reviews.

: [[File:Allow Self Review.PNG]]

: This will be updated to ask instructors if peer-reviews should be visible before self-reviews are completed.

: [[File:Allow Self Review Edited.PNG]]

: When checked the system will check if a student has completed a self-review when they access the scores view. If they have not completed their self-review they will see a message indicating such. If they have completed the review, they will see their peer-reviews as normal.

; Find or create a function that will derive a score from self-reviews and peer-reviews.

: This score should be reflective of the difference between the students self-review score, and their peer's reviews. The purpose behind this score is to teach students to become better at reviewing their own work. By providing a score that reflects how similar their reviews are to their peer's reviews, students receive feedback on their self-assessment abilities. More information on the function creation/decision proccess can be found in the section below.

; Create the different displays for self-reviews

: Instructors and students both have a heat-map view of scores from peer-reviews

: [[File:Instructor Review Heatmap.png]]

:(Instructor view pictured) student's self-review scores will be added to this view, but need to be denoted in a way that makes it easy to see this is a self-review not a peer-review. One potential option is to use an icon that shows this is a self-review score.

: [[File:Instructor Review Heatmap edited.png]]

: Which makes it simple for users to understand that particular column represents the scores from a self-review. Other potential options are to title the column "Self-review", or to highlight/border the column with a particular color (ex. blue), both denoting that the score comes from a self-review. Similar approaches can be taken to differentiate between self-review and peer-review scores in the regular "view scores" page.

== Score Formulas ==
There are many possible methods of deriving a score between self and peer-reviews. Each formula has the same goal, to derive a meaningful score between the peer and self-reviews. Thus, giving feedback to students on their self-reviewing, hopefully increasing their abilities in this area. Several potential approaches are available, each with different pros and cons. The simpler of approaches involves only the self and peer-review scores, and the more complex approaches may involve several other factors.

The simplest approach is to simply take the difference between the average peer-review score, and the self-review score. This rewards the students for good self-reviewing ability, and evenly punishes students for over and underrating themselves. A similar approach is utilized in several other systems, in which when a students self-review score is within 5% of their average peer-review score, they will receive the greater of the two grades. An extension of this can be made to allow a weight to be applied to the difference between the grades, and to allow instructors to manually enter the weight for each assignment.

[[File:Design Doc Naive Formula.PNG]]

A slight change to this approach is to take into account the peer score, so the greater your peers believe you performed, the greater your score is (proportional to the difference between reviews again). However, this, and similar approaches, should be avoided as they are potentially harmful. In the chart below, it can be seen that students with poor performance will be harshly punished for poor performance in their peer-reviews, their self and peer-review derived grade, and finally in their instructor grade. The self and peer-review derived grade should only be indicative of the students ability to self-review, not of the quality of the project.

[[File:Design Doc Take Score Into Account.PNG]]

Another approach is to derive the grade based on the percent difference between the self-review score, and the peer-review scores. This results in a smooth gradient which does not punish students as harshly as the previous two approaches. However, at low scores students will see greater swings in this derived grade.

[[File:Design Doc Pecentage Diff.PNG]]

These previous approaches involve only the peer and self-review scores, and are the simple approaches to the problem, the following options involve greater complexity, but seek to better solve the problem at hand.

The SPARK approach is discussed [https://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/343/258 here.] Which results in a score that should accurately reflect the relation between self and peer scores. The scores here will need to be adjusted to match the grading scheme of the project, a score of 1 is equivalent to a 100%, and the difference between 1 and the score is representative of a lower grade, regardless of the score being above or below 1. This method also includes a second score titled SAPA, Self-Assessment to Peer-Assessment factor. This score requires teams to evaluate each team member on the same self-review rubric. After this is completed team member's self-review scores are divided by their average teammate-review score to create the SAPA. A SAPA of > 1.0 indicates that the student rated themselves higher than their team mates did, a score of less than 1.0 indicates that students underrated themselves compared to their peers.

[[File:Design Doc Spark Formula.PNG]]

Two other approaches involving teammate participation are ranking and FCS(Fair Contribution Scoring).

In the ranking approach, students are asked to rank their team, including themselves, from 1 to n (where n is equal to team size), where 1 is the highest score, and n is the lowest score. Students are then assessed based on their average rank from teammates and the rank they awarded themselves. This approach is simple, but it forgoes the rubric approach that garners greater self-reflection in specific areas.

In the FCS approach, students are given a finite number of points to award to their teammates, including themselves. The more points that a person receives, the greater their perceived work contribution. At the end of the project students are assessed based on their average number of points received, and the points awarded to themselves. While the approach is very similar to the ranking approach there are important differences. With this approach students will need to weight the contributions of each member on a percentage of points basis, rather than on a, teammate A vs teammate B overall basis.

{| class="wikitable"
|+ Approach Pros and Cons
! Approach
! Pros
! Cons
|-
! Simple approaches
| Simple to implement, scoring is quick, direct feedback based only on student's self-review ability
| Does not take into account teammate assessments, has potential pitfalls at low peer-review scores
|-
! SPARK approach
| Simple to implement, includes SAPA which shows how self-reviews compare to teammate-reviews
| A separate metric must be calculated, stored, and displayed, team-reviews will slow the process
|-
! Ranking approach
| A very simple approach, can be implemented similar to the bidding page
| Loss of some self-reflection without a rubric
|-
! FCS approach
| A slightly improved ranking approach, scores may be more accurate than just ranking
| Loss of some self-reflection without a rubric
|}

== Testing ==
Test cases are as follows:
* Peer-reviews cannot be viewed before a self-review is completed.
* Peer-reviews can be viewed immediately when self-reviews are not required.
* The derived scores can be saved to the database.
* Self-review scores are displayed with peer-review scores
* Student can only see their own self-review

== Files Changed ==

== Reference ==
[https://github.com/YourSignificantOtter/expertiza Our repo link]

our pull request link

Edward F. Gehringer, "Self-assessment to improve learning and evaluation," ASEE Annual Conference and Exposition, Columbus, OH, June 25–28, 2017

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T02:35:54Z

Naherrma: /* Introduction and Problem Statement */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T02:31:45Z

Naherrma: /* Reference */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T02:31:12Z

Naherrma: /* Score Formulas */

File:Design Doc Pecentage Diff.PNG

2018-04-14T02:30:44Z

Naherrma: uploaded a new version of "File:Design Doc Pecentage Diff.PNG"

File:Design Doc Spark Formula.PNG

2018-04-14T02:28:58Z

Naherrma: uploaded a new version of "File:Design Doc Spark Formula.PNG"

File:Design Doc Take Score Into Account.PNG

2018-04-14T02:28:35Z

Naherrma: uploaded a new version of "File:Design Doc Take Score Into Account.PNG"

File:Design Doc Naive Formula.PNG

2018-04-14T02:27:50Z

Naherrma: uploaded a new version of "File:Design Doc Naive Formula.PNG"

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T02:14:35Z

Naherrma: /* Planned Approach */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-14T02:05:50Z

Naherrma: updated self review to self-review, same for peer review to peer-review

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-13T23:41:13Z

Naherrma: /* Reference */

== Introduction and Problem Statement ==
Expertiza has the ability to require self reviews on projects, however, these self review scores do not have much functionality. In the current system, self review scores are created and stored, but are not used afterwards. This project seeks to improve upon the self review system, and extend its functionality. The main points of this project are as follows:
* Display the self review scores in the "View Scores" page juxtaposed with peer review scores.
* Display the self review scores in the heat map of peer reviews, with some way to denote they are a different type of review.
* Create a method to calculate a linked score between self and peer reviews. The closer the self review is to the peer review, the greater the score. Display this score on the "View Scores" page.
* Ensure that users cannot view peer reviews until they complete their self review.

== Planned Approach ==
; Allow instructors to decide if students should be allowed to view peer reviewers before completing their self-review.

: Currently when instructors create a new assignment, or edit an existing one, they have the option to require self-reviews.

: [[File:Allow Self Review.PNG]]

: This will be updated to ask instructors if peer-reviews should be visible before self-reviews are completed.

: [[File:Allow Self Review Edited.PNG]]

: When checked the system will check if a student has completed a self-review when they access the scores view. If they have not completed their self-review they will see a message indicating such. If they have completed the review, they will see their peer-reviews as normal.

; Find or create a function that will derive a score from self-reviews and peer-reviews.

: This score should be reflective of the difference between the students self-review score, and their peer's reviews. The purpose behind this score is to teach students to become better at reviewing their own work. By providing a score that reflects how similar their reviews are to their peer's reviews, students receive feedback on their self-assessment abilities. More information on the function creation/decision proccess can be found in the section below.

; Create the different displays for self reviews

: Instructors and students both have a heat-map view of scores from peer-reviews

: [[File:Instructor Review Heatmap.png]]

:(Instructor view pictured) student's self-review scores will be added to this view, but need to be denoted in a way that makes it easy to see this is a self-review not a peer-review. One potential option is to use an icon that shows this is a self-review score.

: [[File:Instructor Review Heatmap edited.png]]

: Which makes it simple for users to understand that particular column represents the scores from a self-review. Other potential options are to title the column "Self-review", or to highlight/border the column with a particular color (ex. blue), both denoting that the score comes from a self-review.

== Score Formulas ==
Several potential formulas to calculate the derived score between self and peer reviews were discussed.
A naive approach is to find the difference between the max score and the difference between self and peer reviews.
[[File:Design Doc Naive Formula.PNG]]
A slight change to this approach is to take into account the peer score, so the greater your peers believe you performed, the greater your score is (proportional to the difference between reviews still). However, this approach can prove to be harmful. Below it can be seen that even when you perfectly reflect the score of your peers your grade can be very low. This approach punishes students twice for low peer review grades, and it should be avoided.
[[File:Design Doc Take Score Into Account.PNG]]
A third approach is discussed [https://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/343/258 here.] Which results in a score that should accurately reflect the relation between self and peer scores. However, it can again be seen that when peer review scores are low students are punished twice unless they can accurately reflect their peer's reviews.
[[File:Design Doc Spark Formula.PNG]]
A fourth approach is to derive the grade based on the percent difference between the self review score, and the peer review scores. This results in a smooth gradient which does not punish students as harshly as the previous two approaches. However, at low scores students will see greater swings in this derived grade.
[[File:Design Doc Pecentage Diff.PNG]]

== Testing ==
Test cases are as follows:
* Peer reviews cannot be viewed before a self review is completed.
* Peer reviews can be viewed immediately when self reviews are not required.
* The derived scores can be saved to the database.
* Self review scores are displayed with peer review scores
* Student can only see their own self review

== Files Changed ==

== Reference ==
[https://github.com/YourSignificantOtter/expertiza Our repo link]

our pull request link

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-13T23:40:00Z

Naherrma: /* Planned Approach */

File:Instructor Review Heatmap edited.png

2018-04-13T23:38:03Z

Naherrma:

File:Instructor Review Heatmap.png

2018-04-13T23:36:20Z

Naherrma:

File:Allow Self Review Edited.PNG

2018-04-13T23:23:24Z

Naherrma:

File:Allow Self Review.PNG

2018-04-13T23:22:06Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-13T23:00:09Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-10T15:57:50Z

Naherrma: /* Score Formulas */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-10T15:53:01Z

Naherrma:

File:Design Doc Pecentage Diff.PNG

2018-04-10T15:50:29Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-09T02:20:08Z

Naherrma:

File:Design Doc Spark Formula.PNG

2018-04-09T02:16:15Z

Naherrma:

File:Design Doc Take Score Into Account.PNG

2018-04-09T02:08:24Z

Naherrma:

File:Design Doc Naive Formula.PNG

2018-04-09T02:03:45Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-09T00:55:13Z

Naherrma: /* Reference */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-09T00:54:50Z

Naherrma: /* Introduction */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-09T00:05:39Z

Naherrma:

== Introduction ==
Expertiza has the ability to require self reviews on projects, however, these self review scores do not have much functionality. In the current system, self review scores are created and stored, but are not used afterwards. This project seeks to improve upon the self review system, and extend its functionality. The main points of this project are as follows:
* Display the self review scores in the "View Scores" page juxtaposed with peer review scores.
* Display the self review scores in the heat map of peer reviews, with some way to denote they are a different type of review.
* Create a method to calculate a linked score between self and peer reviews. The closer the self review is to the peer review, the greater the score. Display this score on the "View Scores" page.
* Ensure that users cannot view peer reviews until they complete their self review.

== Planned Approach ==
# Remove ability to see peer reviews before submitting a self review
# Create/find a function to link peer and self review scores
# Implement function in a method that will be used when a self review is submitted
# Create the different displays for self reviews

== Testing ==
Test cases are as follows:
* Peer reviews cannot be viewed before a self review is completed.
* Peer reviews can be viewed immediately when self reviews are not required.
* The derived scores can be saved to the database.
* Self review scores are displayed with peer review scores

== Files Changed ==

== Reference ==
Our Repo link
our pull request link

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-08T23:55:04Z

Naherrma: /* Introduction */

CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades

2018-04-08T23:39:57Z

Naherrma: Created page with "== Introduction == Expertiza allows for user to perform both self and peer reviews. Currently these two scores are not connected, this project seeks to add a link between them. T..."

== Introduction ==
Expertiza allows for user to perform both self and peer reviews. Currently these two scores are not connected, this project seeks to add a link between them. The grade derived from a completed self review should be reflective of how closely that review matches reviews from peers.

== Planned Approach ==
Add formula

== Improvements to Self Reviews ==
Make them better

== Reference ==
Our Repo link
our pull request link

MainPage

2018-04-08T23:35:45Z

Naherrma:

==Expertiza==
* [[Expertiza documentation]]

* [[CSC/ECE 517 Summer 2008]]
* [[CSC/ECE 517 Fall 2010]]
* [[CSC/ECE 517 Fall 2011]]
* [[CSC/ECE 517 Fall 2012]]
* [[CSC/ECE 517 Fall 2013]]
* [[CSC/ECE 517 Fall 2014]]
* [[CSC/ECE 517 Fall 2015]]
* [[CSC/ECE 517 Fall 2016]]
* [[CSC/ECE 517 Spring 2014]]
* [[CSC/ECE 517 Spring 2015]]
* [[CSC/ECE 517 Spring 2016]]
* [[CSC/ECE 517 Spring 2017]]
* [[CSC/ECE 517 Fall 2017]]
* [[CSC/ECE 517 Spring 2018- Project Juniper:Bookmark Enhancements]]
* [[CSC/ECE 517 Spring 2018- Project E1803: Introducing a Student View for Instructors]]
* [[CSC/ECE 517 Spring 2018- Project E1804: OSS project Yellow: Topic management]]
* [[CSC/ECE_517_Spring_2018- Project E1805: Convolutional data extraction from Github]]
* [[CSC/ECE 517 Spring 2018- Project E1808: Refactor review_mapping_controller.rb]]
* [[CSC/ECE 517 Spring 2018- Project E1810: Show sample submissions and reviews]]
* [[CSC/ECE 517 Spring 2018/E1814 Write unit tests for collusion cycle.rb]]
* [[CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering]]
* [[CSC/ECE 517 Spring 2018 - E1800: Add past-due assignments to task list]]
* [[CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb]]
* [[CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches ]]
* [[CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)]]
* [[CSC/ECE 517 Spring 2018/E1813 Test Menu Items Model]]
* [[CSC/ECE 517 Spring 2018- Project E1816: Visualization for Instructors]]
* [[CSC/ECE 517 Spring 2018- Project E1817: Adding Student-generated Questions to Rubric]]
* [[CSC/ECE 517 Spring 2018- Project E1822: Extend the functionality of badging]]
* [[CSC/ECE 517 Spring 2018- Project E1815: Improvements to review grader]]
* [[CSC/ECE 517 Spring 2018- Project E1824: Let course staff as well as students do reviews]]
* [[CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades]]
* [[CSC 456 Spring 2011|CSC 456 Spring 2012]]
* [[ECE 633]]
* [[KCU]]
* [[Progress reports]]

==Application Behavior==
* [[Grading]]

==Metaprogramming==
* [[CSC/ECE_517_Spring_2013/ch1b_1k_hf|Lecture on Metaprogramming]]

==Development==

''Expertiza now has a Java dependency, so the machine you are using to develop Expertiza on should have the JVM installed.''

* [[Setting Up a Development Machine]]
* [[Creating a Linux Development Environment for Expertiza - Installation Guide]]
* [[Using git and github for projects]]
* [[Using heroku to deploy your projects]]
* [[How to Begin a Project from the Current Expertiza Repository]]
* [[Git]]
* [[How to Change a User's Password on a Development Machine]]
* [[Debugging Rails]]
* [http://rajanalwan.com/ui_guidelines/ Design Template]

==Production==
* [[Deploying to Production]]
* [[Downloading Production Data]]
* [[Accessing the Production Server]]

==Testing==
* [[Using Cucumber with Expertiza]]
* [[Rails Testing Overview]]
* [[Expertiza Continuous Integration]]

==References==
* [[Object-Oriented Design and Programming]]

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-04-04T03:51:55Z

Naherrma:

== Introduction ==

[http://expertiza.ncsu.edu/ Expertiza] is a web based open source tool developed and maintained by current and past students of [https://en.wikipedia.org/wiki/North_Carolina_State_University North Carolina State University]. Instructors

== Problem Statement ==
The objective for this project is: Integrate Github metrics into Expertiza. The tasks required to complete this were broken down as follows.
# Retrieve data on a Git pull request
## Calculate meaningful metrics from the data for teams and individual students
## Display metrics to instructors
# Store metrics in the Expertiza database

== Previous Design ==
[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1783_Convolutional_data_extraction_from_Github Previous] work done on this project utilized the Github statistics API, which was unable to retrieve information on individual students. The design was able to accurately fetch and display data covering a team of contributors, but could not create and display individual metrics to instructors.

== Proposed Design ==
The proposed design utilizes the new [https://developer.github.com/v4/ GraphQL Git API] which allows for simple extraction of metrics on a pull request. The API call returns information on multiple commits with only a single fetch command. The returned data is then parsed into team level information and individual level information. Next, the metrics are calculated from the data and displayed to the user.

== Improvements ==
By manipulating GraphQL API v4, we don't need to pull out all the information from GitHub, which is not customizable. Instead, we can appoint what kind of data we want to fetch so that it's much more efficient than Github Pull Requests API and also we don't need to parse bulk information in order to get the portion we need.

[[File:Screen Shot 2018-04-02 at 9.16.22 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.40 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.56 PM.png]]

After having the data we want, we start to divide them based on dates and users so that reviewer can understand the contribution of each member clearly.
[[File:Screen Shot 2018-04-02 at 9.17.05 PM.png]]

==Results==
The output of this module shows the users github data in various metrics through charts and tables. The images below show the metrics gathered, and the charts created when looking at [https://github.com/expertiza/expertiza/pull/1159 this pull request].
[[File:E1805_Commits_Line_Additions.png]]
[[File:E1805_Line_Deletions_Files_Changed.png]]
[[File:E1805_Full_Information.png]]

==Environment Setup==
To test this project manually on your local machine, you need to:

1. Clone from this repository https://github.com/rpot/expertiza/tree/E1805GithubDataExtraction

2. Go to your GitHub account -> Settings -> Developer Settings -> Personal access tokens -> Generate new token -> name the token and check the repo and admin:repo_hook option -> copy the token generated

3. Run GITHUB_ACCESS_TOKEN=<token> rails server, then you can manually test the functionality by going to Assignments as instructor6 -> view submissions -> Show Submission Records -> Show Github Data

Important note: If the project is added to the production environment of Expertiza, an access token must be created for the system and stored internally by some method!
== Testing Plan==
Tests for this project include testing two methods in app/models/github_datum.rb and one method in app/controllers/github_data_controller.rb by spec/models/github_datum_spec.rb and spec/controllers/github_data_controller_spec.rb.

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-04-04T03:45:56Z

Naherrma:

== Introduction ==

[http://expertiza.ncsu.edu/ Expertiza] is a web based open source tool developed and maintained by current and past students of [https://en.wikipedia.org/wiki/North_Carolina_State_University North Carolina State University]. Instructors

== Problem Statement ==
The objective for this project is: Integrate Github metrics into Expertiza. The tasks required to complete this were broken down as follows.
# Retrieve data on a Git pull request
## Calculate meaningful metrics from the data for teams and individual students
## Display metrics to instructors
# Store metrics in the Expertiza database

== Previous Design ==
[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1783_Convolutional_data_extraction_from_Github Previous] work done on this project utilized the Github statistics API, which was unable to retrieve information on individual students. The design was able to accurately fetch and display data covering a team of contributors, but could not create and display individual metrics to instructors.

== Proposed Design ==
The proposed design utilizes the new [https://developer.github.com/v4/ GraphQL Git API] which allows for simple extraction of metrics on a pull request. The API call returns information on multiple commits with only a single fetch command. The returned data is then parsed into team level information and individual level information. Next, the metrics are calculated from the data and displayed to the user.

== Improvements ==
By manipulating GraphQL API v4, we don't need to pull out all the information from GitHub, which is not customizable. Instead, we can appoint what kind of data we want to fetch so that it's much more efficient than Github Pull Requests API and also we don't need to parse bulk information in order to get the portion we need.

[[File:Screen Shot 2018-04-02 at 9.16.22 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.40 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.56 PM.png]]

After having the data we want, we start to divide them based on dates and users so that reviewer can understand the contribution of each member clearly.
[[File:Screen Shot 2018-04-02 at 9.17.05 PM.png]]

==Results==
The output of this module shows the users github data in various metrics through charts and tables
[[File:E1805_Commits_Line_Additions.png]]
[[File:E1805_Line_Deletions_Files_Changed.png]]
[[File:E1805_Full_Information.png]]

==Environment Setup==
To test this project manually on your local machine, you need to:

1. Clone from this repository https://github.com/rpot/expertiza/tree/E1805GithubDataExtraction

2. Go to your GitHub account -> Settings -> Developer Settings -> Personal access tokens -> Generate new token -> name the token and check the repo and admin:repo_hook option -> copy the token generated

3. Run GITHUB_ACCESS_TOKEN=<token> rails server, then you can manually test the functionality by going to Assignments as instructor6 -> view submissions -> Show Submission Records -> Show Github Data

Important note: If the project is added to the production environment of Expertiza, an access token must be created for the system and stored internally by some method!
== Testing Plan==
Tests for this project include testing two methods in app/models/github_datum.rb and one method in app/controllers/github_data_controller.rb by spec/models/github_datum_spec.rb and spec/controllers/github_data_controller_spec.rb.

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-04-04T03:43:28Z

Naherrma:

== Introduction ==

[http://expertiza.ncsu.edu/ Expertiza] is a web based open source tool developed and maintained by current and past students of [https://en.wikipedia.org/wiki/North_Carolina_State_University North Carolina State University]. Instructors

== Problem Statement ==
The objective for this project is: Integrate Github metrics into Expertiza. The tasks required to complete this were broken down as follows.
# Retrieve data on a Git pull request
## Calculate meaningful metrics from the data for teams and individual students
## Display metrics to instructors
# Store metrics in the Expertiza database

== Previous Design ==
[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1783_Convolutional_data_extraction_from_Github Previous] work done on this project utilized the Github statistics API, which was unable to retrieve information on individual students. The design was able to accurately fetch and display data covering a team of contributors, but could not create and display individual metrics to instructors.

== Proposed Design ==
The proposed design utilizes the new [https://developer.github.com/v4/ GraphQL Git API] which allows for simple extraction of metrics on a pull request. The API call returns information on multiple commits with only a single fetch command. The returned data is then parsed into team level information and individual level information. Next, the metrics are calculated from the data and displayed to the user.

== Improvements ==
By manipulating GraphQL API v4, we don't need to pull out all the information from GitHub, which is not customizable. Instead, we can appoint what kind of data we want to fetch so that it's much more efficient than Github Pull Requests API and also we don't need to parse bulk information in order to get the portion we need.

[[File:Screen Shot 2018-04-02 at 9.16.22 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.40 PM.png]]
[[File:Screen Shot 2018-04-02 at 9.16.56 PM.png]]

After having the data we want, we start to divide them based on dates and users so that reviewer can understand the contribution of each member clearly.
[[File:Screen Shot 2018-04-02 at 9.17.05 PM.png]]

==Results==
The output of this module shows the users github data in various metrics through charts and tables
[[File:E1805_Commits_Line_Additions.png]]
[[File:E1805_Line_Deletions_Files_Changed.png]]
[[File:E1805_Full_Information.png]]

==Environment Setup==
To test this project manually on your local machine, you need to:

1. Clone from this repository https://github.com/rpot/expertiza/tree/E1805GithubDataExtraction

2. Go to your GitHub account -> Settings -> Developer Settings -> Personal access tokens -> Generate new token -> name the token and check the repo and admin:repo_hook option -> copy the token generated

3. Run GITHUB_ACCESS_TOKEN=<token> rails server, then you can manually test the functionality by going to Assignments as instructor6 -> view submissions -> Show Submission Records -> Show Github Data

== Testing Plan==
Tests for this project include testing two methods in app/models/github_datum.rb and one method in app/controllers/github_data_controller.rb by spec/models/github_datum_spec.rb and spec/controllers/github_data_controller_spec.rb.

File:E1805 Line Deletions Files Changed.png

2018-04-04T03:43:05Z

Naherrma:

File:E1805 Full Information.png

2018-04-04T03:42:34Z

Naherrma:

File:E1805 Commits Line Additions.png

2018-04-04T03:40:21Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-28T00:19:39Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-28T00:11:08Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-28T00:09:54Z

Naherrma:

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-28T00:08:22Z

Naherrma:

== Introduction ==

[http://expertiza.ncsu.edu/ Expertiza] is a web based open source peer reviewing tool developed and maintained by current and past students of [https://en.wikipedia.org/wiki/North_Carolina_State_University North Carolina State University]. Peer review is a great way for a student to learn how to approach a project and get ideas for their own projects. Currently, there is no way for a student to view another student's work, unless they are reviewing the other student's work.

== Problem Statement ==
The objective for this project is:
# Integrate Github metrics into Expertiza
# Retrieve data on a Git pull request
## Calculate meaningful metrics from the data for teams and individual students
## Display metrics to instructors
# Store metrics in the Expertiza database

== Previous Design ==
[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E1783_Convolutional_data_extraction_from_Github Previous] work done on this project utilized the Github statistics API, which was unable to retrieve information on individual students. The design was able to accurately fetch and display data covering a team of contributors, but could not create and display individual metrics to instructors.

== Proposed Design ==

== Improvements ==

== Testing ==

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-28T00:05:11Z

Naherrma:

MainPage

2018-03-27T22:00:39Z

Naherrma:

==Expertiza==
* [[Expertiza documentation]]

* [[CSC/ECE 517 Summer 2008]]
* [[CSC/ECE 517 Fall 2010]]
* [[CSC/ECE 517 Fall 2011]]
* [[CSC/ECE 517 Fall 2012]]
* [[CSC/ECE 517 Fall 2013]]
* [[CSC/ECE 517 Fall 2014]]
* [[CSC/ECE 517 Fall 2015]]
* [[CSC/ECE 517 Fall 2016]]
* [[CSC/ECE 517 Spring 2014]]
* [[CSC/ECE 517 Spring 2015]]
* [[CSC/ECE 517 Spring 2016]]
* [[CSC/ECE 517 Spring 2017]]
* [[CSC/ECE 517 Fall 2017]]
* [[CSC/ECE 517 Spring 2018- Project Juniper:Bookmark Enhancements]]
* [[CSC/ECE 517 Spring 2018- Project E1803: Introducing a Student View for Instructors]]
* [[CSC/ECE 517 Spring 2018- Project E1804: OSS project Yellow: Topic management]]
* [[CSC/ECE_517_Spring_2018- Project E1805: Convolutional data extraction from Github]]
* [[CSC/ECE 517 Spring 2018- Project E1808: Refactor review_mapping_controller.rb]]
* [[CSC/ECE 517 Spring 2018- Project E1810: Show sample submissions and reviews]]
* [[CSC/ECE 517 Spring 2018/E1814 Write unit tests for collusion cycle.rb]]
* [[CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering]]
* [[CSC/ECE 517 Spring 2018 - E1800: Add past-due assignments to task list]]
* [[CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb]]
* [[CSC 456 Spring 2011|CSC 456 Spring 2012]]
* [[ECE 633]]
* [[KCU]]
* [[Progress reports]]

==Application Behavior==
* [[Grading]]

==Metaprogramming==
* [[CSC/ECE_517_Spring_2013/ch1b_1k_hf|Lecture on Metaprogramming]]

==Development==

''Expertiza now has a Java dependency, so the machine you are using to develop Expertiza on should have the JVM installed.''

* [[Setting Up a Development Machine]]
* [[Creating a Linux Development Environment for Expertiza - Installation Guide]]
* [[Using git and github for projects]]
* [[Using heroku to deploy your projects]]
* [[How to Begin a Project from the Current Expertiza Repository]]
* [[Git]]
* [[How to Change a User's Password on a Development Machine]]
* [[Debugging Rails]]
* [http://rajanalwan.com/ui_guidelines/ Design Template]

==Production==
* [[Deploying to Production]]
* [[Downloading Production Data]]
* [[Accessing the Production Server]]

==Testing==
* [[Using Cucumber with Expertiza]]
* [[Rails Testing Overview]]
* [[Expertiza Continuous Integration]]

==References==
* [[Object-Oriented Design and Programming]]

CSC/ECE 517 Spring 2018- Project E1805: Convolutional data extraction from Github

2018-03-27T21:58:32Z

Naherrma: Created page with "Test of the page"

Test of the page