Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T11:17:37Z

Zwang18: /* Javascript Event Handler */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.

[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage frontend AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

====chart object and chart====
Chart object is an ruby object created by calling Chart.new(type, data, [option], [width]). It is just a ruby object representing the chart.

In order to display the chart, you need to render '_chart.erb', this partial will create html and javascript code for a given chart (represented by the chart object).

====code responsibilities====
If one do not want to 'customize' the chart, see ''render_sample.html.erb'' for quick setup examples. If one want to 'customize' charts, or want to understand ''index.html.erb'', here is something need to know:
* '''views/analytic/_chart.erb''': render a chart represented by given chart object. it handles all javascript headers, javascript and HTML for you. see ''render_sample.html.erb'' for detail.
* '''views/analytic/_static.erb''': JSON format chart grid theme descriptor, this is a global settings which will affect all chart.
* '''model/analytic/chart.rb'''
** '''''get_id_str''''': get the unique id of the container element of chart, only meaningful when all charts are created by rendering "_chart.erb".
** '''''test_data''''': Create test data for chart.
** '''''data_template''''': Creates chart settings template. All chart object created by Chart.new() is based on this template. One can overwrite this method to get full control of the chart to be created. (Note: This is a global settings which will affect all chart with corresponding chart type.)
** '''''initialize''''': create a chart object, parameters are:
*** ''type'': specify the type of the graph(:bar, :line, :pie, :scatter)
*** ''data'': data to be represented by the chart. see ''render_sample.html.erb'' for details.
*** ''option'': optional parameter, one can specify some frequently used chart options here, default value is nil. see ''render_sample.html.erb'' for details.
*** ''witdh'': optional parameter, specify the width of the chart, like "500px" or "80%", default is "100%". see ''render_sample.html.erb'' for details.

====DOM tree in index.html.erb====
Enable nulti-chart support by specifing ''multi_chart = true'' at the beginning of the <script> section. Default value is false.

Structure of the tree looks like this:

[[File:DOM Tree.PNG]]

All charts will be in one container, selector should be '#draw_area'. Each chart and its corresponding options is in its own container with selector '.chart_X' where X is the index of that chart. Within each chart container, selector '.option' will return an container contains all end-user-specified options (chart type, comparison scope, courses, asssignments, etc. each in a container with '.tr_X' selector), selector '.chart' will give the chart container.

====Javascript Event Handler====
Each <input> element is attached an onchange handler. When the handler is triggered, it will:

1. check if the layout should change(i.e. the end user changes comparison scope from "assignment" to "team")

If so, trigger an AJAX request to ruby controller to fetch the new options to be displayed and erase the old layout if necessary.

2. check if the end user has specify all required input to display a chart. If so, trigger an AJAX request to ruby controller to fetch the chart object and display it; if not, erase the chart(if there's already one).

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T11:16:57Z

Zwang18: /* Javascript Event Listener */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.

[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage frontend AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

====chart object and chart====
Chart object is an ruby object created by calling Chart.new(type, data, [option], [width]). It is just a ruby object representing the chart.

In order to display the chart, you need to render '_chart.erb', this partial will create html and javascript code for a given chart (represented by the chart object).

====code responsibilities====
If one do not want to 'customize' the chart, see ''render_sample.html.erb'' for quick setup examples. If one want to 'customize' charts, or want to understand ''index.html.erb'', here is something need to know:
* '''views/analytic/_chart.erb''': render a chart represented by given chart object. it handles all javascript headers, javascript and HTML for you. see ''render_sample.html.erb'' for detail.
* '''views/analytic/_static.erb''': JSON format chart grid theme descriptor, this is a global settings which will affect all chart.
* '''model/analytic/chart.rb'''
** '''''get_id_str''''': get the unique id of the container element of chart, only meaningful when all charts are created by rendering "_chart.erb".
** '''''test_data''''': Create test data for chart.
** '''''data_template''''': Creates chart settings template. All chart object created by Chart.new() is based on this template. One can overwrite this method to get full control of the chart to be created. (Note: This is a global settings which will affect all chart with corresponding chart type.)
** '''''initialize''''': create a chart object, parameters are:
*** ''type'': specify the type of the graph(:bar, :line, :pie, :scatter)
*** ''data'': data to be represented by the chart. see ''render_sample.html.erb'' for details.
*** ''option'': optional parameter, one can specify some frequently used chart options here, default value is nil. see ''render_sample.html.erb'' for details.
*** ''witdh'': optional parameter, specify the width of the chart, like "500px" or "80%", default is "100%". see ''render_sample.html.erb'' for details.

====DOM tree in index.html.erb====
Enable nulti-chart support by specifing ''multi_chart = true'' at the beginning of the <script> section. Default value is false.

Structure of the tree looks like this:

[[File:DOM Tree.PNG]]

All charts will be in one container, selector should be '#draw_area'. Each chart and its corresponding options is in its own container with selector '.chart_X' where X is the index of that chart. Within each chart container, selector '.option' will return an container contains all end-user-specified options (chart type, comparison scope, courses, asssignments, etc. each in a container with '.tr_X' selector), selector '.chart' will give the chart container.

====Javascript Event Handler====
Each <input> element is attached an onchange handler. When the handler is triggered, it will
1. check if the layout should change(i.e. the end user changes comparison scope from "assignment" to "team")
If so, trigger an AJAX request to ruby controller to fetch the new options to be displayed and erase the old layout if necessary.
2. check if the end user has specify all required input to display a chart. If so, trigger an AJAX request to ruby controller to fetch the chart object and display it; if not, erase the chart(if there's already one).

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T11:01:46Z

Zwang18: /* DOM tree in index.html.erb */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.

[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage frontend AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

====chart object and chart====
Chart object is an ruby object created by calling Chart.new(type, data, [option], [width]). It is just a ruby object representing the chart.

In order to display the chart, you need to render '_chart.erb', this partial will create html and javascript code for a given chart (represented by the chart object).

====code responsibilities====
If one do not want to 'customize' the chart, see ''render_sample.html.erb'' for quick setup examples. If one want to 'customize' charts, or want to understand ''index.html.erb'', here is something need to know:
* '''views/analytic/_chart.erb''': render a chart represented by given chart object. it handles all javascript headers, javascript and HTML for you. see ''render_sample.html.erb'' for detail.
* '''views/analytic/_static.erb''': JSON format chart grid theme descriptor, this is a global settings which will affect all chart.
* '''model/analytic/chart.rb'''
** '''''get_id_str''''': get the unique id of the container element of chart, only meaningful when all charts are created by rendering "_chart.erb".
** '''''test_data''''': Create test data for chart.
** '''''data_template''''': Creates chart settings template. All chart object created by Chart.new() is based on this template. One can overwrite this method to get full control of the chart to be created. (Note: This is a global settings which will affect all chart with corresponding chart type.)
** '''''initialize''''': create a chart object, parameters are:
*** ''type'': specify the type of the graph(:bar, :line, :pie, :scatter)
*** ''data'': data to be represented by the chart. see ''render_sample.html.erb'' for details.
*** ''option'': optional parameter, one can specify some frequently used chart options here, default value is nil. see ''render_sample.html.erb'' for details.
*** ''witdh'': optional parameter, specify the width of the chart, like "500px" or "80%", default is "100%". see ''render_sample.html.erb'' for details.

====DOM tree in index.html.erb====
Enable nulti-chart support by specifing ''multi_chart = true'' at the beginning of the <script> section. Default value is false.

Structure of the tree looks like this:

[[File:DOM Tree.PNG]]

All charts will be in one container, selector should be '#draw_area'. Each chart and its corresponding options is in its own container with selector '.chart_X' where X is the index of that chart. Within each chart container, selector '.option' will return an container contains all end-user-specified options (chart type, comparison scope, courses, asssignments, etc. each in a container with '.tr_X' selector), selector '.chart' will give the chart container.

====Javascript Event Listener====

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

File:DOM Tree.PNG

2013-04-29T10:48:16Z

Zwang18:

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T10:41:40Z

Zwang18: /* Framework Design */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.

[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage frontend AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

====chart object and chart====
Chart object is an ruby object created by calling Chart.new(type, data, [option], [width]). It is just a ruby object representing the chart.

In order to display the chart, you need to render '_chart.erb', this partial will create html and javascript code for a given chart (represented by the chart object).

====code responsibilities====
If one do not want to 'customize' the chart, see ''render_sample.html.erb'' for quick setup examples. If one want to 'customize' charts, or want to understand ''index.html.erb'', here is something need to know:
* '''views/analytic/_chart.erb''': render a chart represented by given chart object. it handles all javascript headers, javascript and HTML for you. see ''render_sample.html.erb'' for detail.
* '''views/analytic/_static.erb''': JSON format chart grid theme descriptor, this is a global settings which will affect all chart.
* '''model/analytic/chart.rb'''
** '''''get_id_str''''': get the unique id of the container element of chart, only meaningful when all charts are created by rendering "_chart.erb".
** '''''test_data''''': Create test data for chart.
** '''''data_template''''': Creates chart settings template. All chart object created by Chart.new() is based on this template. One can overwrite this method to get full control of the chart to be created. (Note: This is a global settings which will affect all chart with corresponding chart type.)
** '''''initialize''''': create a chart object, parameters are:
*** ''type'': specify the type of the graph(:bar, :line, :pie, :scatter)
*** ''data'': data to be represented by the chart. see ''render_sample.html.erb'' for details.
*** ''option'': optional parameter, one can specify some frequently used chart options here, default value is nil. see ''render_sample.html.erb'' for details.
*** ''witdh'': optional parameter, specify the width of the chart, like "500px" or "80%", default is "100%". see ''render_sample.html.erb'' for details.

====DOM tree in index.html.erb====
Enable nulti-chart support by specifing ''multi_chart = true'' at the beginning of the <script> section. Default value is false.

Structure of the tree looks like this:

All charts will be in one container, selector should be '#draw_area'. Each chart and its corresponding options is in its own container with selector '.chart_X' where X is the index of that chart. Within each chart container, selector '.option' will return an container contains all end-user-specified options (chart type, comparison scope, courses, asssignments, etc), selector '.chart' will give the chart container. Unless all required options are specified, chart will not be displayed.

====Javascript Event Listener====

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T10:21:36Z

Zwang18: /* javascript libraries */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.
[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage frontend AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

====chart object and chart====
Chart object is an ruby object created by calling Chart.new(type, data, [option], [width]). It is just a ruby object representing the chart.

In order to display the chart, you need to render '_chart.erb', this partial will create html and javascript code for a given chart (represented by the chart object).

====code responsibilities====
If one do not want to 'customize' the chart, see ''render_sample.html.erb'' for quick setup examples. If one want to 'customize' charts, or want to understand ''index.html.erb'', here is something need to know:
* '''views/analytic/_chart.erb''': render a chart represented by given chart object. it handles all javascript headers, javascript and HTML for you. see ''render_sample.html.erb'' for detail.
* '''views/analytic/_static.erb''': JSON format chart grid theme descriptor, this is a global settings which will affect all chart.
* '''model/analytic/chart.rb'''
** '''''get_id_str''''': get the unique id of the container element of chart, only meaningful when all charts are created by rendering "_chart.erb".
** '''''test_data''''': Create test data for chart.
** '''''data_template''''': Creates chart settings template. All chart object created by Chart.new() is based on this template. One can overwrite this method to get full control of the chart to be created. (Note: This is a global settings which will affect all chart with corresponding chart type.)
** '''''initialize''''': create a chart object, parameters are:
*** ''type'': specify the type of the graph(:bar, :line, :pie, :scatter)
*** ''data'': data to be represented by the chart. see ''render_sample.html.erb'' for details.
*** ''option'': optional parameter, one can specify some frequently used chart options here, default value is nil. see ''render_sample.html.erb'' for details.
*** ''witdh'': optional parameter, specify the width of the chart, like "500px" or "80%", default is "100%". see ''render_sample.html.erb'' for details.

====DOM tree in index.html.erb====
Enable nulti-chart support by specifing ''multi_chart = true'' at the beginning of the <script> section. Default value is false.

Structure of the tree looks like this:

All charts will be in one container, selector should be '#draw_area'. Each chart and its corresponding options is in its own container with selector '.chart_X' where X is the index of that chart. Within each chart container, selector '.option' will return an container contains all end-user-specified options (chart type, comparison scope, courses, asssignments, etc), selector '.chart' will give the chart container. Unless all required options are specified, chart will not be displayed.

====Javascript Event Listener====

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T07:11:21Z

Zwang18: /* View */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.
[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
====javascript libraries====
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

CSC/ECE 517 Spring 2013/OSS E734

2013-04-29T07:10:00Z

Zwang18: /* New Implementation */

Developers:
*Hao Liu (hliu11@ncsu.edu)
*Zhuowei Wang (zwang18@ncsu.edu)
*Chunxue Yang (cyang14@ncsu.edu)

[https://docs.google.com/a/ncsu.edu/document/d/11YTjxFXFR13vJ769yFBbqn9qOueK2Ktz0Pd-gyS2W5U/edit# Write-up of This Topic.]

This page is the design documentation for Expertiza project E734 conducted in the CSC/ECE 517 Spring 2003 class at North Carolina State University. Learn more about Expertiza by visiting its main
page: http://wikis.lib.ncsu.edu/index.php/Expertiza

__TOC__

= Introduction =
The Expertiza project is system for using peer review to create reusable learning objects. Students do different assignments; then peer review selects the best work in each category, and assembles it to create a single unit.

The process of reviewing includes reviews, re-reviews and meta reviews of assignments submitted by students for a course. This method of evaluation also inherently suggests that there will be vagaries in the way reviews have been performed.

This project aim to provide a way for the users to examine various type of statistic about the courses, assignments and reviews.

== Objective ==
The original objective of the project is to extend the Analytic project E732 from previous semester to incorporate new features.
Upon examination the implementation of E732 we decided to start from scratch and create a developer friendly and easily extendable framework for providing analytic data and displaying analytic data.

To fulfill our purpose of providing a better experience for Expertiza users, we have several different aspects of to focus on.

The primary focus of this project is on reviews and the related aspects that go hand in hand with reviews like meta reviews, re-reviews etc. To be more specific, we will be working on an advanced search interface in which different criteria can be selected more flexibly to make a comparison for each review or meta review.

Moreover, we will dedicate in developing a more dynamic and clear chart graph. In our project, JavaScript libraries such as jQuery.js (GPL), highchart.js(CC by 3.0) are included to support the dynamic behaviors of the chart graph. Multi chart areas are supported by Dom Tree.

= Graphical User Interface (GUI) Design =
== Original Design ==
[[File:Original_analytic_gui.png]]

In the original GUI design of analytic the user can select from a list of cryptically worded data comparison:
*Review: Question Text And Review Comment vs Assignment
*Review: Question Count And Review Comment vs Assignment
*Review: Sub Question Count And Review Comment vs Assignment
*Average Review Parameters And Student Experience vs Assignment
*Average Metareview Parameters And Student Experience vs Assignment
*Assignment Strategy And Review Parameter

Besides the unfathomable meaning options, the design does not provide much room for the user of the page to explore and discover interesting correlations between data. Since the objective is to enable user to discover correlations between data, this implementation does not really meet the requirement.

After selecting a graph type the user can add courses to the graph. Upon selection the all of the assignments from the course are added to the graph than the user can un-select the assignments from a drop down menu. Once a course is selected there's no way to remove a course from the selection.

There are no option for selecting how the data are presented, data are displayed as a bar graph which restrict the type of data that can be displayed.

The specific numeric value for the graph are displayed below the graph as a chart.

== New Design ==
[[File:New_analytic_gui.png]]
=== Graph Type and Comparison Scope Selection ===
When the user arrive at the analytic page they are prevented with two options: Graph Type and Comparison Scope.

[[File:New_analytic_gui_1.png]]

First, in the new design we would like to offer the user and developer more options for displaying the data, instead of the restriction of only using bar graph.

Graph type let the user select what kind of graph do they want the data to be displayed currently there are four different options:
*Bar Graph - useful for displaying simple numeric comparisons (i.e: average scores, number of participants)
[[File:New_analytic_gui_bar.png]]
*Line Graph - useful for displaying comparison of distributions (i.e: grade distribution for assignments)
[[File:New_analytic_gui_line.png]]
*Scatter Plot - useful for displaying raw data points (i.e: grades for all of the student for assignments)
[[File:New_analytic_gui_scatter.png]]
*Pie Chart - useful for displaying percentage related data (i.e: percentage of grade distribution for an single assignment)
[[File:New_analytic_gui_pie.png]]

Second, we would like to offer the user more scope of comparison instead restricting to only assignment wise comparison

Comparison scope give user options to select what do they want to compare:Course, Assignment, Team

=== Comparison Object Selection===
After the user select the graph type and comparison scope the user are presented with options to select comparison object and comparison data. These options changes depending on the graph type and comparison type.

Comparison object depends on what comparison scope the user selected. For example:

If the user select course wise comparison scope than the user will be presented with a check-box list of courses that they are allow to view.

[[File:New_analytic_gui_2_1.png‎]]

However if the user select assignment wise comparison scope the user will be presented with drop down menu with list of courses that they are allow to view and once the user select the course that they are interested in the are presented with a check-box list of assignments that are within the courses.

[[File:New_analytic_gui_2_2.png‎]]

User can select multiple object that they want to compare and the graph will update dynamically.

=== Comparison Data Selection ===
Options for comparison data depends on both comparison scope and graph type. A check-box list will be displayed containing the appropriate data type that are available for the selected graph type and selected comparison scope.

'''Development Note:''' Currently we only finish the options related to for bar graph, however the framework for adding options related to other type of graph has been completed.

[[File:New_analytic_gui_3.png]]

The user can select multiple type of graph data and the graph will be updated dynamiclly

=== Graph Display ===
Once all of the selections are complete a graph containing the data will be rendered. We would like to retain the ability to display the numeric value for the data, however table containing all of the data in the graph is not the most efficient way to to percent the data instead in the new implementation the numeric value for the data will be display when the user hover the curer over the graph.

[[File:New_analytic_gui_4.png]]

= Framework Design =
Architecture of the Expertiza Analytic Framework is based on the model-view-controller design pattern used in Rails application.
[[File:Analytic_framwork_archtecture.png]]

== Model ==
There are two tasks that analytic require the model to perform: data mining and data formating.

=== Data Mining ===
The data mining operations performed by model classes gathers the data requested by the controller.

==== Original Implementation ====
In the original implementation of analytic, data mining methods use to extract the data are scattered between the assignment model class and controller. There are several problems with this method of implementation.

First, assignment model is crucial class for the correct operation of the Expertiza and the data mining methods are not related to the normal operation that the model performs this reduces the readability and maintainability of the model.

Second, the data mining methods seem to recursively traverse through the data structure hierarchy level by level (for example: to obtain average grades for assignment, the original implementation do assignment->for each teams-> for each responses->score->grade) in-order to obtain the data that they need. This design violates basic object oriented programming principle, instead the data mining methods should be layered and methods should be added to the corresponding models to avoid this behavior.

==== New Implementation ====
The new implementation of analytic the data mining methods are implemented as mixin modules for the corresponding models. The modules are gathered into the /models/analytic folder. The modules contains the analytic methods for the model that they relates to. The analytic modules only calls the methods that are defined in model that they corresponds to and methods defined in other analytic module.

This implementation creates a clear separation between the normal model operation and data mining operation; abstract away the complexity of the structure of the underlying databases; improves the readability for the developer.

==== Related Files ====
Analytic modules and their corresponding models:
*expertiza/app/models/analytic/course_analytic.rb => expertiza/app/models/course.rb
*expertiza/app/models/analytic/assignment_analytic.rb => expertiza/app/models/assignment.rb
*expertiza/app/models/analytic/assignment_team_analytic.rb => expertiza/app/models/assignment_team.rb
*expertiza/app/models/analytic/questionnaire_analytic.rb => expertiza/app/models/questionnaire.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/response_analytic.rb => expertiza/app/models/response.rb
*expertiza/app/models/analytic/question_analytic.rb => expertiza/app/models/question.rb
*expertiza/app/models/analytic/score_analytic.rb => expertiza/app/models/score.rb

=== Data Formatting ===
The data formatting operations take the gathered data and formatted turn it into the format that are require by the methods that draws the charts that display the data.

==== Original Implementation ====
The original implementation of analytic have the controller handles data formatting function. In the MVC structure used in rails application, formatting the data is not a responsibility for the controller.

==== New Implementation ====
The new implementation of analytic a Chart class was created to handle the formatting of data. The Chart class serve as a interface between the java script based front-end in view and the ruby model back-end. The controller passes the graph type and data into the Chart class and the Chart class will handle all formatting and return the data that controller can pass into view.

The example for the input data format and how to use the Chart class will be provided in later section.

==== Related files ====
*expertiza/app/helpers/chart.rb

'''Development Note:''' currently the Chart class located in the helper folder, this is not the proper location for the file. The file should be move to the models folder and rename to analytic.rb.

== View ==
In analytic view displays options for the user, send user input to the controller and presents graph in a particular format, triggered by a controller's decision based on the user input.

=== Original Implementation ===
The original implementation of analytic leverages the google chart ruby gem to generate the graph in a static fashion. The graph data type that the user can request are hard coded into the view with lots of switch statements and if else statements to handle different cases. This method of implementation which limits what the user can see and dramatically increases the difficult for adding new data options that user can see. According to the documentation for the MVC architecture used in Rails, the decision for what to display should be handle by the controller.

Using google chart ruby gem is an acceptable implementation however the google chart gem have limited capabilities and a somewhat outdated look. Further more, the gem it might have the risk running into compatibly issues with future Rails version if the gem is not kept up to date.

=== New Implementation ===
===javascript libraries===
For our new implementation of analytic we decided to move to jQuery and hightchart.

jQuery is a powerful, popular, and well maintained webpage AJAX library for javascript, its documentation is available here [http://api.jquery.com/]. For the UI part, we use highchart which is a jQuery compatible chart library, the documentation: [http://api.highcharts.com/highcharts]. Licenses for these two libraries are GPL and CC by 3.0 for non-commercial usage.

The original jQuery library used by Expertiza is version 1.4, we have it upgraded to version 1.9 in this project, locates at /public/javascripts/jquery.js and highchart is at /public/javascripts/highcharts.js.

== Controller ==

=== Method Overview ===
methods with post fix _list (graph_data_type_list, course_list, assignment_list, team_list) are use to populate the different selection in view.

get_graph_data_bundle method is use to generate the data for the graph in view

bar_chart_data, line_chart_data, scatter_chart_data, pie_chart_data located in analytic helper (expertiza/app/helpers/analytic_helper.rb) are the interface function between the analytic modules and chart class

=== Access Control ===
Goldberg menu system provided control for who can access the analytic page however it does not control what courses are displayed once they are in the analytic page. By assigning permission of administer assignment to the controller in Goldberg menu system we restricted the access of analytic to user with role of super-admin, admin, instructor and teaching assistant. However we would like the user to only be able to see the courses that they are associated with. To achieve that we the method associated_courses was added to the course helper (expertiza/app/helpers/course_helper.rb)

The associated_courses method returns a list of courses depending on the user's role. For super-admin and admin it will return all of the courses that are in the database. For instructor it will return the courses that belongs to the instructor. For teaching assistant it will return all the courses that they are TA-ing.

The course_list method in the analytic controller uses this method to determine which list should be listed on the course selection section of the view.

=== Adding New Option for the Data Type ===
The method graph_data_type_list is use to return a list of appropriate options for the data type selection in view.

@available_data_type is a hash of comparison_scope and graph_type with method names in the analytic modules initialized in the init method.

all of the methods within the analytic modules will be automatically pull into the @available_data_type[comparison_scope] (i.e: @available_data_type[:course] will contain all of the methods defined in the CourseAanlytic module) @available_data_type[graph_type] contain the subset of methods that in the analytic modules that are suited for the particular graph_type.

The method uses the cross section of @available_data_type[comparison_scope] and @available_data_type[graph_type] to determine the appropriated data type options in view

To add new data type simply add a the data mining method in the corresponding analytic module than add the name of the method as a string into @available_data_type for the appropriate graph_type

= Developer Resources =
[https://github.com/q115288339/expertiza github repository for the project]
the content of this repository might change suggest using [https://github.com/expertiza/expertiza the main repository for expertiza] if the branch has already been merged

[http://www.youtube.com/watch?v=nGRxQtqaqOo&list=PLqv3LmpyB8wB76ZZjKnHBGfYqSBBnVt81 Youtube playlist about E734]

== Development Environment Setup ==
*follow the instruction on expertiza wiki page to setup the development environment http://wikis.lib.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_the_Expertiza_Application
''note:'' you might need to delete the automated_metareview table from the database before you can successfully run db:migrate
*launch expertiza server (ruby script/server)
*visit the page from your browser (0.0.0.0:3000)
*login as admin (username: admin, password: password)
*setup access permission for the controller
**on the menu bar navigates to administration->setup->controller/action
**under the missing tab you should see analytic, click on it and select administer_assignment for the permission than click add
**under the application tab click on analytic and add index and render_sample to the action list
*add analytic to menu bar for easy access (alternatively just enter 0.0.0.0:3000/analytic in your browser)
**on the menu bar navigates to administration->setup->menu editor
**add an menu option for analytic/index
''note:'' not all of the courses and assignments in scrubbed database are real (with actual data), in order to get courses and assignments that have real data you might want to consider impersonate a real instructor in the scrubbed data base (i.e: Dr.Geringer, user id: efg) this can be done by navigating to manage...->impersonate_user from the menu bar.

== Rendering Example ==
For the benefit of future developers we provided examples on how to use the Chart class (chart.rb) to format data for the chart and how to use the jQuery stubs (_chart.html.erb) to render the chart

The example is located at expertiza/app/views/analytic/render_sample

To access the webpage created by this example visit 0.0.0.0:3000/analytic/render_sample (assuming you follow the setup instruction above)

= Future Works =
Sorted by task difficulty
*Move expertiza/app/helpers/chart.rb to expertiza/app/model and rename to analytic.rb
*Rename Chart class to Analytic
*Unit test
*Add data mining methods for meta-review
*Add options for displaying raw data point related data using scatter plot(for example: raw grades)
*Come up with other interesting data for the comparison scope
*Improve the performance of data mining methods especially score related methods.
*Add user comparisons (comparing student with student)
*Give control of options in the of graph type selection and comparison scope selection drop down to controller (require java script experience)
*Implement a static version of the index page using ruby (require java script experience)
*Strengthen access control by checking if the current user have permission to access to the data that they requested

'''IMPORTANT NOTE:''' There is a bug in the get_average_score function in the response module class WATCH OUT!

User:Zwang18/Writting assignment 1g

2013-02-14T06:02:15Z

Zwang18: /* Frontend */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results. Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play and CodeIgniter are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database. The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP. The client, on web applications is a web browser that runs HTML generated by the application layer. The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
*jQuery
*ExtJS
*Prototype

Compared to the backend web framework, the number of languages that can be used by frontend framework is limited. This is because the frontend frameworks are executed at client side, therefore it has compatibility problem. Although both VBScript and javascript can be used for frontend coding, Javascript is an undisputed standard language for frontend framework.

Frontend framework mainly focus on two tasks. The data stream between server and client, and the user interface. It offers various efficient method to find and change the DOM model, and also strengthen the ability to manage asynchronous request and responses.

Frontend framework is usually very compact cause large frontend framework will significantly slowdown web page loading speed. Therefore, usually the main library is small and with only limited functionality. But one can always statically or dynamically load plug-ins for frontend framework on demand. In such way, frontend framework can support many features while keep the loading speed fast.

User:Zwang18/Writting assignment 1g

2013-02-14T06:01:52Z

Zwang18: /* Frontend */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results. Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play and CodeIgniter are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database. The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP. The client, on web applications is a web browser that runs HTML generated by the application layer. The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
*jQuery
*ExtJS
*Prototype

Compared to the backend web framework, the number of languages that can be used by frontend framework is limited. This is because the frontend frameworks are executed at client side, therefore it has compatibility problem. Although both VBScript and javascript can be used for frontend coding, Javascript is an undisputed standard language for frontend framework.

Frontend framework mainly focus on two tasks. The data stream between server and client, and the user interface. It offers various efficient method to find and change the DOM model, and also strengthen the ability to manage asynchronous request and responses.

Frontend framework is usually very compact cause large frontend framework will significantly slowdown web page loading speed. Therefore, usually the main library is small and with only limited functionality.But one can always statically or dynamically load plug-ins for frontend framework on demand. In such way, frontend framework can support many features while keep the loading speed fast.

User:Zwang18/Writting assignment 1g

2013-02-14T06:00:47Z

Zwang18: /* Frontend */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results. Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play and CodeIgniter are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database. The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP. The client, on web applications is a web browser that runs HTML generated by the application layer. The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
*jQuery
*ExtJS
*Prototype

Compared to the backend web framework, the number of languages that can be used by frontend framework is limited. This is because the frontend frameworks are executed at client side, therefore it has compatibility problem. Although both VBScript and javascript can be used for frontend coding, Javascript is an undisputed standard language for frontend framework.

Frontend framework mainly focus on two tasks. The data stream between server and client, and the user interface. It offers various efficient method to find and change the DOM model, and also the strengthen the ability to manage asynchronous request and responses.

Frontend framework is usually very compact cause large frontend framework will significantly slowdown web page loading speed. Therefore, usually the main library is small and with only limited functionality.But one can always statically or dynamically load plug-ins for frontend framework on demand. In such way, frontend framework can support many features while keep the loading speed fast.

User:Zwang18/Writting assignment 1g

2013-02-13T02:08:46Z

Zwang18: /* Backend Types */

User:Zwang18/Writting assignment 1g

2013-02-13T01:05:43Z

Zwang18: /* Backend Types */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results.[5] Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play, CodeIgniter, and Struts2 are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database. The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP. The client, on web applications is a web browser that runs HTML generated by the application layer. The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
*jQuery
*ExtJS
*Prototype

User:Zwang18/Writting assignment 1g

2013-02-13T01:05:06Z

Zwang18: /* Frontend */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results.[5] Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play, CodeIgniter, and Struts2[6] are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database.[7][8][9][10] The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP.[11] The client, on web applications is a web browser that runs HTML generated by the application layer.[12][13] The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.[14][15]

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
*jQuery
*ExtJS
*Prototype

User:Zwang18/Writting assignment 1g

2013-02-12T23:46:20Z

Zwang18: /* Backend Types */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===MVC===
MVC (Model–view–controller) is a popular type of framework. The MVC pattern aims at separating the data model with business rules from the user interface.

The data model focuses on the structure of persistent stored data and relationship between them. The view model mainly deal with the user interface hence HTML code and display content, it often work with template. The controller handles the processing logic and algorithms for the data.

This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.

===Push-based vs. pull-based===

Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results.[5] Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play, CodeIgniter, and Struts2[6] are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.

===Three-tier organization===

In Three Tier Organization, applications are structured around three physical tiers: client, application, and database.[7][8][9][10] The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP.[11] The client, on web applications is a web browser that runs HTML generated by the application layer.[12][13] The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.[14][15]

===CMS===

CMS (Content Management System) stands for a type of web application which aimed at organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. It is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.

Unlike other type of framework, CMS system focus on content management and the functionality as well as the logic is almost fixed. Therefore, there are many pre-programed CMS system which allows user without programming experience to set up a web application at the cost of degree of customization. In such CMS system, features can be added through plug-ins.

==Frontend==
===jQuery===
===ExtJS===
===Prototype===

User:Zwang18/Writting assignment 1g

2013-02-12T23:24:07Z

Zwang18: /* Backend Types */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
===Model–view–controller (MVC)===
Many frameworks follow the model–view–controller (MVC) architectural pattern to separate the data model with business rules from the user interface. This is generally considered a good practice as it modularizes code, promotes code reuse, and allows multiple interfaces to be applied. In Web applications, this permits different views to be presented, such as web pages for humans, and web service interfaces for remote applications.
===Push-based vs. pull-based===
Most MVC frameworks follow a push-based architecture also called "action-based". These frameworks use actions that do the required processing, and then "push" the data to the view layer to render the results.[5] Struts, Django, Ruby on Rails, Symfony, Yii, Spring MVC, Stripes, Play, CodeIgniter, and Struts2[6] are good examples of this architecture. An alternative to this is pull-based architecture, sometimes also called "component-based". These frameworks start with the view layer, which can then "pull" results from multiple controllers as needed. In this architecture, multiple controllers can be involved with a single view. Lift, Tapestry, JBoss Seam, JavaServer Faces, and Wicket are examples of pull-based architectures.
===Three-tier organization===
In Three Tier Organization, applications are structured around three physical tiers: client, application, and database.[7][8][9][10] The database is normally an RDBMS. The Application contains the business logic, running on a server and communicates with the client using HTTP.[11] The client, on web applications is a web browser that runs HTML generated by the application layer.[12][13] The term must not be confused with MVC. Unlike in three-tier architecture, it is considered a good practice to keep business logic away from the controller, the "middle layer" in MVC.[14][15]
===Content management systems===
Main article: Content management system
In web application framework Content management is the way of organizing, categorizing, and structuring the information resources like text, images, documents, multimedia files such as audio or video files so that they can be stored, published, and edited with ease and flexibility. and A content management system (CMS) is used to collect, manage, and publish content, storing the content either as components or whole documents, while maintaining dynamic links between components.
Some projects that have historically been termed content management systems have begun to take on the roles of higher-layer Web application frameworks. For instance, Drupal's structure provides a minimal core whose function is extended through modules that provide functions generally associated with web application frameworks. The Joomla platform provides a set of APIs to build Web and command-line applications. However, it is debatable whether "management of content" is the primary value of such systems, especially when some, like SilverStripe, provide an object-oriented MVC framework. Add-on modules now enable these systems to function as full-fledged applications beyond the scope of content management. They may provide functional APIs, functional frameworks, coding standards, and many of the functions traditionally associated with Web application frameworks.

==Frontend==
===jQuery===
===ExtJS===
===Prototype===

User:Zwang18/Writting assignment 1g

2013-02-12T23:22:43Z

Zwang18: /* Database */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
There are various backend web framework
====CMS====
Plug-ins
====MVC====
====3-Tiers====
====Push-based vs. pull-based====

==Frontend==
===jQuery===
===ExtJS===
===Prototype===

User:Zwang18/Writting assignment 1g

2013-02-12T23:17:10Z

Zwang18: /* CGI Standrad */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standard===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
There are various backend web framework
====CMS====
Plug-ins
====MVC====
====3-Tiers====
====Push-based vs. pull-based====

==Frontend==
===jQuery===
===ExtJS===
===Prototype===
==Database==
===SQLite===
===MySQL===
===Cloud SQL===
===BigTable===
===mongoDB===

User:Zwang18/Writting assignment 1g

2013-02-12T23:17:01Z

Zwang18: /* CGI and PHP */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI Standrad===
'''CGI''' (Common Gateway Interface) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
There are various backend web framework
====CMS====
Plug-ins
====MVC====
====3-Tiers====
====Push-based vs. pull-based====

==Frontend==
===jQuery===
===ExtJS===
===Prototype===
==Database==
===SQLite===
===MySQL===
===Cloud SQL===
===BigTable===
===mongoDB===

User:Zwang18/Writting assignment 1g

2013-02-12T23:15:23Z

Zwang18: /* Backend */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI and PHP===
'''CGI''' (Common Gateway Interface) and '''PHP''' (Hypertext Preprocessor) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go
*'''Sina App Engine''': Java Python, PHP Script

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
There are various backend web framework
====CMS====
Plug-ins
====MVC====
====3-Tiers====
====Push-based vs. pull-based====

==Frontend==
===jQuery===
===ExtJS===
===Prototype===
==Database==
===SQLite===
===MySQL===
===Cloud SQL===
===BigTable===
===mongoDB===

User:Zwang18/Writting assignment 1g

2013-02-12T23:13:16Z

Zwang18: /* Features */

==Introduction==
'''Web framework''' is a [[software framework]] that is designed to support the development of web applications like dynamic website, web service APIs. It frees developer from lower layer details and provide tools to accelerate the development. A framework usually correspond to one or more languages and types of Database.

Except for standard libraries, web framework often provide libraries for frequently used functions in web applications and hide low level details to the developer. It also has all required protocols well implemented, i.e. HTTP(s) and all supporting protocols. Except the standard library, most web framework will include libraries for frequently used functions and modules by default, like: [[URL Encoding]], [[HTTP Request header parsing]], [[URL mapping]], [[session module]] and [[Database module]].

In addition, [[Template]], [[Caching]], [[Load balance]], [[URL Fetch]], [[DoS Protection]], [[Frontend Channel]], [[Mail]], [[Open Authorization]], [[DOM management]], [[XMPP]] are also commonly seen features in most modern web framework.

==Web application process==
Most web applications are based on HTTP(s) protocol, where client and server uses Request-Response method to exchange information. In a typical round, client sends information encoded into a HTTP request header, server will respond with a HTML page, Json or XML data. The response is usually real-time generated based on the request.

Usually server only respond to requests and is not able to start a connection to client, unless certain hack like Javascript Channel is used.

===Static===
The static server is a direct implementation of HTTP protocol. In such model, every request corresponds to a file on server, the server daemon fetch the correspond file and send the file directly to client as is.

Therefore, as its name indicates, the response content is not dynamically generated.

[[Image:Static.JPG]]

===CGI and PHP===
'''CGI''' (Common Gateway Interface) and '''PHP''' (Hypertext Preprocessor) are naive implementation of dynamic web server. The response content are dynamically generated according to input parameters which is sent to server with the request header.

In this way, the server can customize the response based on the request parameters and client information.

These kind of web framework have a simple script interpreter or compiled server program in order to generate the response. However such framework only provides an executable environment for the response generator, usually without further functions.

Furthermore, these interpreter or program usually don't have strong protection or isolation from the operating system. That means the input parameters must be examined very carefully, otherwise an intruder may able to execute any command with the privilege of the interpreter or server program.

[[Image:CGI_PHP.JPG]]

===Modern framework===
The basic task of a modern framework is still generate dynamic response according to the request and corresponding parameters. However, it provides many feature to make the task easier and more secure.

The framework may have build in modules to provide certain task like session management, templating and it may also provide interfaces which allow the server program to communicate with third parties upon user request. Those communication may use protocols other than HTTP(s), like XMPP or SMTP.

Modern framework runs not only at backend, but also at frontend: providing features like DOM management, asynchronous communications and two-way communications.

A modern framework includes protecting mechanism which prevents intruders from accessing the host operating system directly. Only through certain modules or interface can a service program access resources in operation system. therefore it is more secure.

[[Image:web_framwork.JPG]]

==Features==
One of the advantages of using a web framework is that most frequently used features are already implemented and can be used easily through pre-defined interfaces.

Some of the features are listed below:
===Database management===
Most web framework have database module which allow the server program to provide persistent storage to data. This is considered a basic feature that should be provided by a web framework.

It is important to store and query data on server so that the response is not uniquely depend on the corresponding request. The response can now be generated upon previous session content even session between the server and other clients.

===Session management===
Session management is another basic feature in web framework.

HTTP protocol is designed to be stateless which makes it hard to track a client's identity. Therefore, without Session management, the server cannot provide continuous user-specific service, because the server cannot keep the client identity between requests.

Session also provide a temporary memory storage between associated requests. It is much faster and cheaper to store temporary data using session modules instead of database.

===Template===
A web application that responds with an HTML document often consist of static parts (Markups, i.e. Nodes) and dynamic parts(content, i.e. Text Nodes). A template is a file that contains static parts of HTML document and also specify where and how the dynamic content should mixed in to the static part.

By using template feature, the developer can avoid many lines of output function call, the only propose of those function calls are just trying to output a constant string that consist the static HTML code.

===Caching===
Web caching is the caching of web documents in order to reduce bandwidth usage, server load, and perceived latency. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

Some web frameworks has multi-level caching and even can control caches stored at the client side.

===Gzip===
HTTP protocols tends to transfer readable documents(json, HTML, XML) directly through internet. Thus HTTP packets have the potential to be compressed in order to save the bandwidth of the server.

Gzip is a compressing mechanism supported by HTTP protocol to compress the content of HTTP packet, it is supported by more and more web frameworks nowadays.

===Channel===
A channel can provide a persistent connection between client and server, allowing the client to send messages to JavaScript runs in client browser in real time without the use of polling.

This is useful for applications that are designed to update the user about new information immediately or where user input is immediately broadcast to other users. Without channel feature, the only way to do so may be client explicitly polling the server which results in high latency and high server load.

===Email and XMPP===
Web applications with user interfaces usually use HTTP protocol. However, there are web applications that uses other protocols like SMTP (Email services), XMPP (instant message services), etc.

It would be useful to have the ability for a HTTP based web application to cooperate with other protocols. And it is important to do so through build-in modules of the web framework instead of access those functionality though operating system because for security reasons.

===Open Authentication and URLFetch ===
Open Authentication protocols like [[OAuth]] and [[Open ID]] are protocols that allow a user to grant a third party limited permission to access a web application on his/her behalf, without sharing credentials with the third party.

Although most Open Authentication Protocols are based on HTTP(s). These protocols often have too many detail and procedures for a developer to write a compatible authentication adapter. Thus it becomes popular for web frameworks to have such modules build-in.

URL fetch enables the server to generate dynamic content based on HTTP(s) resources on other hosts or communicate with other HTTP(s) hosts over the Internet.requests.and communicate with other

===Load Balance and Data Synchronization===
Some web framework like [[Google Appengine]] are designed to work on multiple servers. Hence it becomes important to balance loads between servers and synchronize data between servers.

These kind of web framework often have load balance and data synchronization modules so that these trivial things can be handled by the framework automatically and efficiently.

===Security and Server Logs===
Security is another important aspect of web framework. Web frameworks often provides many mechanisms to protect the server from deny of services attack, SQL injection and prevent control flow escaped to operating system.

Some web application frameworks also come with authentication and authorization module, which is usually more secure compared to a user implemented authentication module.

Web framework should also provide a event log which makes it easier to debug and locate the security flaw.

==List of Frameworks==
===Backend===
*'''ASP''': VB Script
*'''Rails''': Ruby
*'''GRails''': Java
*'''CakePHP''': PHP Script
*'''Kohana''': PHP Script
*'''Symfony''': PHP Script
*'''Yii''': PHP Script
*'''Django''': Python
*'''TurboGears''': Python
*'''web.py''': Python
*'''web.go''': Go
*'''Google Appengine''': Java, Python, Go

===Frontend===
*'''jQuery'''
*'''ExtJS'''
*'''Prototype'''

==Backend Types==
There are various backend web framework
====CMS====
Plug-ins
====MVC====
====3-Tiers====
====Push-based vs. pull-based====

==Frontend==
===jQuery===
===ExtJS===
===Prototype===
==Database==
===SQLite===
===MySQL===
===Cloud SQL===
===BigTable===
===mongoDB===

User:Zwang18/Writting assignment 1g

2013-02-12T22:49:06Z

Zwang18: /* Backend */

User:Zwang18/Writting assignment 1g

2013-02-12T22:48:57Z

Zwang18: /* Backend */

User:Zwang18/Writting assignment 1g

2013-02-12T22:48:25Z

Zwang18: /* Backend */

User:Zwang18/Writting assignment 1g

2013-02-12T22:47:39Z

Zwang18: /* Backend */

User:Zwang18/Writting assignment 1g

2013-02-12T22:47:18Z

Zwang18: /* languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:46:37Z

Zwang18: /* Backend and corresponding programming languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:46:18Z

Zwang18: /* List of Frameworks and programming languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:42:11Z

Zwang18: /* Features */

User:Zwang18/Writting assignment 1g

2013-02-12T22:30:55Z

Zwang18: /* List of Frameworks and programming languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:25:42Z

Zwang18: /* List of Frameworks and programming languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:19:43Z

Zwang18: /* Email and XMPP */

User:Zwang18/Writting assignment 1g

2013-02-12T22:18:10Z

Zwang18: /* Security */

User:Zwang18/Writting assignment 1g

2013-02-12T22:10:05Z

Zwang18: /* List of Frameworks and corresponding Languages */

User:Zwang18/Writting assignment 1g

2013-02-12T22:09:48Z

Zwang18: /* Frameworks */

User:Zwang18/Writting assignment 1g

2013-02-12T22:07:52Z

Zwang18: /* Open Authentication and URLFetch */

User:Zwang18/Writting assignment 1g

2013-02-12T22:07:34Z

Zwang18: /* Open Authentication and URLFetch */

User:Zwang18/Writting assignment 1g

2013-02-12T21:50:34Z

Zwang18: /* Features */

User:Zwang18/Writting assignment 1g

2013-02-12T21:41:33Z

Zwang18: /* Load Balance and Data Synchronize */

User:Zwang18/Writting assignment 1g

2013-02-12T21:38:43Z

Zwang18: /* Load Balance */

User:Zwang18/Writting assignment 1g

2013-02-12T21:35:10Z

Zwang18: /* Session management */

User:Zwang18/Writting assignment 1g

2013-02-12T21:34:54Z

Zwang18: /* Session management */

User:Zwang18/Writting assignment 1g

2013-02-12T21:27:29Z

Zwang18: /* Session management */

User:Zwang18/Writting assignment 1g

2013-02-12T21:25:10Z

Zwang18: /* Database */

User:Zwang18/Writting assignment 1g

2013-02-12T21:24:28Z

Zwang18: /* Session management */

User:Zwang18/Writting assignment 1g

2013-02-12T21:23:34Z

Zwang18: /* Features */

User:Zwang18/Writting assignment 1g

2013-02-12T21:03:43Z

Zwang18: /* Modern framework */

File:Web framwork.JPG

2013-02-12T20:49:49Z

Zwang18: MF

User:Zwang18/Writting assignment 1g

2013-02-12T20:42:45Z

Zwang18: /* CGI and PHP */

User:Zwang18/Writting assignment 1g

2013-02-12T20:34:34Z

Zwang18: /* Modern framework */