Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-23T19:41:10Z

Ylu31: /* Architecture */

<big>S1503. Sahana Eden Mobile Client</big>

=Introduction=
==Purpose==
This project aims at adding new functions to the EdenMobile client app to serve users better.These functions will allow mobile users to add new shelters and hospitals, get information of hospitals and shelters offline/online, get GPS location of device, and provide multilingual support.

==Background==
===What’s Sahana Eden?===
Sahana Eden is an Open Source Humanitarian Platform which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is the latest evolution of Sahana Software which was initially developed by members of the information technology (IT) community in Sri Lanka to provide solutions for the relief effort following the 2004 Indian Ocean Tsunami.
<ref>[http://eden.sahanafoundation.org/ Sahana Eden Instroduction]</ref> 

===What does Sahana Eden do?===
Sahana Eden provides various of functions which are sorted in several different modules. 

'''Organization Registry''' 
*Sahana Eden’s Organization Registry can track what organizations do and organizations also could record their Offices, Warehouse and Field Sites including their locations so they can be mapped as well as links to other modules such as Human Resources, Assets and Inventory. 
'''Project Tracking''' 
*Sahana Eden provides a valuable tool which could tell people what are going on about the project to help organizations responding to disasters know where the greatest needs are and coordinate with others who are engaged in similar work. 
'''Human Resources''' 
*There are contact lists in Sahana Eden which can be access by users to ensure that the right people can be contacted at the right time. 
'''Inventory''' 
*Sahana Eden can be used to manage inventories of items and match requests for items with warehouses and other facilities which have them available. Sahana Eden can support multiple Catalogs of Items as well as providing alternative items to ensure more effective use of supplies. 
'''Assets''' 
*Sahana Eden is able to manage assets which are needed to respond to disasters, track where they are, who they have been assigned to, and what condition they are in. This ensures that assets are used effectively and efficiently. 
'''Assessments''' 
*Sahana Eden can be used to collect and analyze information from assessments to help organizations more effectively plan their disaster management activities. 
'''Scenarios & Events''' 
*To help organizations better plan for disasters, Sahana Eden can be used to plan for different scenarios, including recording what human resources, assets, facilities and tasks will be needed to effectively respond. 
'''Map''' 
*Sahana Eden has fully integrated mapping functionality which allows any location-based data to be visualized on a map. Maps provide situational awareness which is essential when either planning to prepare for or respond to a disaster. 
'''Shelter Management''' 
*When disasters are widespread and result in population displacement, understanding and tracking the landscape of shelters - and the people in them - is a critical activity. The Shelter Registry provides functionality to list and track information on shelters and on the people arriving and departing. 
'''Messaging''' 
*Sahana Eden provides support for messages to be sent to one or Distribution Groups by Email, SMS, Twitter and Google Talk. Interactive messages can also be set up to allow people to send short message queries to Sahana Eden and receive automatic responses. 

===Sahana Eden Architecture===
The basic Sahana Eden architecture is as follows:

<table style="width: 100%; border: 1px dotted rgb(0, 0, 0); border-collapse: collapse;" cellpadding="5"><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Server</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Apache</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Other web servers can also be used, such as Cherokee.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Application</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Sahana Eden</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Application Framework</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Web2Py</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Programming Language</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Python & Java Script</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Database</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, or SQLite</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, and SQLite are supported.Other databases should be usable without major additional work since Web2Py supplies many connectors.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Operating System</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Linux (Debian recommended)</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Windows and Mac OS X are possible, but only recommended for single-user environments.</td>
</tr></table>

==Scope==
We will realize the signup, secure login and logout function, which would add new module into the following files:
*communication.js
*controller.js
*model.js
*view.js
We will make it available in multiple languages environment which will implement the languages fold.
We will add some new items including:“ hospital”, “vehicles” which will be added into the folder named “plugins”. The following file will be added into their corrsponding folders: 
Hospitals:
*mHospital.js (model)
*hospital.js, editHospital.js, hospital.html, hospital.css, editHospital.html (views)
*hospitalController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Vehicles:
*mVehicles.js (model)
*vehicles.js, editVehicles.js, vehicles.html, vehicles.css, editVehicles.html (views)
*vehiclesController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Also, we find some bugs and eliminate them. These could be any possible file in this project.

==Documentation==
former developer discussion about Sahana Oden
*An [https://docs.google.com/a/ncsu.edu/document/d/11bblC7klisYjN7Gjw6hDcs5TZ8uesJD3o2xnM2gI914/edit# overview of all the Sahana Eden projects]
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework basic introduction] about the framework of Edenmobile including application design, source<ref>[http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework Edenmobile Introduction]</ref>
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile#J2ME list] of all the requirements need to be done in this project.
*An [https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/sahana-eden google discussion], explaining former problems developers met and corresponding solutions
*[https://github.com/szhao8/eden Open source code about Sahana Eden]<ref>[https://github.com/szhao8/eden Sahana Eden Github]</ref>
*[https://github.com/szhao8/EdenMobile Open source code about Eden Mobile]<ref>[https://github.com/szhao8/EdenMobile EdenMobile Github]</ref>

=Design=
==Overview==
In this design, we add several modules to the EdenMobile client app to allow mobile users to add new shelters and hospitals. Also, users can see the list of hospitals and shelters both offline and online and we can pass users' location to the nearest medical facilities. At last, a multilingual environment is also supported.
==Requirements==
The major requirements are that:
*Secure login;
*Multiple languages;
*users are able to work offline, and update forms and content when online;
*The initial forms that will be implemented are Hospitals, and Shelters;

==Architecture==

Below is the overview of the Eden Mobile Platform. It uses a Model-View-Controller design pattern as the foundation of the project. And it also introduces a "plugin" mechanism, from which developers can add new modules (pages) to the main menu by adding them to "plugins" folder.

[[File:S1503_Architecture.png‎ |frame|center|Use Case Diagram]]

==Use Cases==

* '''1. Security Log in'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills login form, which includes username, password and Captcha code.
*** Submit form to Sahana Server. 
* '''2. Submit forms to server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to Sahana Server. 
* '''3. Submit forms to local storage'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to local database. 
* '''4. Download new forms from server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Download new forms from server 
* '''5. Synchronize local data with server '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Synchronize local data with server 
* '''6. Multi-Language Support '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Choose a language from settings page 
* The UML use case diagram is shown below:

[[File:S1503_Usecase.png‎ |frame|center|Use Case Diagram]]

=Implementation=
==Add Multilanguage Support==
===Development-before preparation===
This program is hosted by a very experienced software engineering, who set the blueprint of this whole program. So we communicated with him serval times to get a better understanding of the project need and expectation. Also, he introduced the framework he used and showed some little tricky to us on google group discussion. Then we understand a mapping relationship and the implementation details about how to build a multilanguage support.

[[File:mapping.png]]

===Code Changes===
The html button changes in languages.js:

[[File:languagesjs.png]]

The html button changes in languagesController.js:

[[File:languagecontroller.png]]

Code changes in index.js:

[[File:index.png]]

Our basic implementation translation in controller.js function goes like this:

[[File:basic.png]]

languages.xml mapping information:

[[File:xml.png]]

===Multilanguage UI===
After serval times of discussion and communication, we finally realize the following UI. You can click the language you need to change to it. It's very easy-using and user comfortable. The following screenshot is what it looks like.

After clicking the English button, the English UI shows as following:

[[File:English.png]]

After clicking the Chinese button, the Chinese UI shows as following:

[[File:Chinese.png]]

After clicking the French button, the French UI shows as following:

[[File:French.png]]

After clicking the Bosnian button, the Bosnian UI shows as following:

[[File:Bosnian.png]]

=Troubleshooting=

1. When loading Sahana Eden Mobile's main page, browser console will print the following error message:

[[File:S1503_Error1.png‎ |frame|center|underscore-min.map not found error]]

The problem can be solved by downloading the file [https://github.com/jashkenas/underscore/blob/master/underscore-min.map underscore-min.map] and put it in the directory where underscore-min.js is stored. More information about this error can be found [http://stackoverflow.com/questions/18429625/missing-javascript-map-file-for-underscore-js-when-loading-asp-net-web-page here].

2. When adding multi-language module to Sahana Eden Mobile's main page, browser console will print the following error message:
After adding a language plugin module, users can see a new button called "Language", but it cannot be clicked and what makes worse is that the module affect other modules, for example, we cannot new a shelter. The framework is super complex and we are new here. After discussion, we write a email describing our problems to Tom Baker, the host developer, for help.

[[File:problem.png]]

After we received his short reply, we tried to follow his instruction to check some files but still get no idea what's our problem.

[[File:reply.png]]

Finally, we set breakpoints and used chrome's developer tool to help us debug. Fortunately, we found the an object is NULL, which affect the initiation. Then we found that in our controller there is a little error which leaded to this problem. After revised that, it run well now.

=References=
<references/>

File:S1503 Usecase.png

2015-04-23T19:39:42Z

Ylu31: uploaded a new version of "File:S1503 Usecase.png"

File:S1503 Architecture.png

2015-04-23T19:12:30Z

Ylu31: uploaded a new version of "File:S1503 Architecture.png"

File:S1503 Architecture.png

2015-04-23T19:11:15Z

Ylu31:

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-23T19:10:00Z

Ylu31: /* Architecture */

<big>S1503. Sahana Eden Mobile Client</big>

=Introduction=
==Purpose==
This project aims at adding new functions to the EdenMobile client app to serve users better.These functions will allow mobile users to add new shelters and hospitals, get information of hospitals and shelters offline/online, get GPS location of device, and provide multilingual support.

==Background==
===What’s Sahana Eden?===
Sahana Eden is an Open Source Humanitarian Platform which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is the latest evolution of Sahana Software which was initially developed by members of the information technology (IT) community in Sri Lanka to provide solutions for the relief effort following the 2004 Indian Ocean Tsunami.
<ref>[http://eden.sahanafoundation.org/ Sahana Eden Instroduction]</ref> 

===What does Sahana Eden do?===
Sahana Eden provides various of functions which are sorted in several different modules. 

'''Organization Registry''' 
*Sahana Eden’s Organization Registry can track what organizations do and organizations also could record their Offices, Warehouse and Field Sites including their locations so they can be mapped as well as links to other modules such as Human Resources, Assets and Inventory. 
'''Project Tracking''' 
*Sahana Eden provides a valuable tool which could tell people what are going on about the project to help organizations responding to disasters know where the greatest needs are and coordinate with others who are engaged in similar work. 
'''Human Resources''' 
*There are contact lists in Sahana Eden which can be access by users to ensure that the right people can be contacted at the right time. 
'''Inventory''' 
*Sahana Eden can be used to manage inventories of items and match requests for items with warehouses and other facilities which have them available. Sahana Eden can support multiple Catalogs of Items as well as providing alternative items to ensure more effective use of supplies. 
'''Assets''' 
*Sahana Eden is able to manage assets which are needed to respond to disasters, track where they are, who they have been assigned to, and what condition they are in. This ensures that assets are used effectively and efficiently. 
'''Assessments''' 
*Sahana Eden can be used to collect and analyze information from assessments to help organizations more effectively plan their disaster management activities. 
'''Scenarios & Events''' 
*To help organizations better plan for disasters, Sahana Eden can be used to plan for different scenarios, including recording what human resources, assets, facilities and tasks will be needed to effectively respond. 
'''Map''' 
*Sahana Eden has fully integrated mapping functionality which allows any location-based data to be visualized on a map. Maps provide situational awareness which is essential when either planning to prepare for or respond to a disaster. 
'''Shelter Management''' 
*When disasters are widespread and result in population displacement, understanding and tracking the landscape of shelters - and the people in them - is a critical activity. The Shelter Registry provides functionality to list and track information on shelters and on the people arriving and departing. 
'''Messaging''' 
*Sahana Eden provides support for messages to be sent to one or Distribution Groups by Email, SMS, Twitter and Google Talk. Interactive messages can also be set up to allow people to send short message queries to Sahana Eden and receive automatic responses. 

===Sahana Eden Architecture===
The basic Sahana Eden architecture is as follows:

<table style="width: 100%; border: 1px dotted rgb(0, 0, 0); border-collapse: collapse;" cellpadding="5"><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Server</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Apache</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Other web servers can also be used, such as Cherokee.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Application</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Sahana Eden</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Application Framework</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Web2Py</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Programming Language</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Python & Java Script</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Database</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, or SQLite</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, and SQLite are supported.Other databases should be usable without major additional work since Web2Py supplies many connectors.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Operating System</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Linux (Debian recommended)</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Windows and Mac OS X are possible, but only recommended for single-user environments.</td>
</tr></table>

==Scope==
We will realize the signup, secure login and logout function, which would add new module into the following files:
*communication.js
*controller.js
*model.js
*view.js
We will make it available in multiple languages environment which will implement the languages fold.
We will add some new items including:“ hospital”, “vehicles” which will be added into the folder named “plugins”. The following file will be added into their corrsponding folders: 
Hospitals:
*mHospital.js (model)
*hospital.js, editHospital.js, hospital.html, hospital.css, editHospital.html (views)
*hospitalController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Vehicles:
*mVehicles.js (model)
*vehicles.js, editVehicles.js, vehicles.html, vehicles.css, editVehicles.html (views)
*vehiclesController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Also, we find some bugs and eliminate them. These could be any possible file in this project.

==Documentation==
former developer discussion about Sahana Oden
*An [https://docs.google.com/a/ncsu.edu/document/d/11bblC7klisYjN7Gjw6hDcs5TZ8uesJD3o2xnM2gI914/edit# overview of all the Sahana Eden projects]
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework basic introduction] about the framework of Edenmobile including application design, source<ref>[http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework Edenmobile Introduction]</ref>
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile#J2ME list] of all the requirements need to be done in this project.
*An [https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/sahana-eden google discussion], explaining former problems developers met and corresponding solutions
*[https://github.com/szhao8/eden Open source code about Sahana Eden]<ref>[https://github.com/szhao8/eden Sahana Eden Github]</ref>
*[https://github.com/szhao8/EdenMobile Open source code about Eden Mobile]<ref>[https://github.com/szhao8/EdenMobile EdenMobile Github]</ref>

=Design=
==Overview==
In this design, we add several modules to the EdenMobile client app to allow mobile users to add new shelters and hospitals. Also, users can see the list of hospitals and shelters both offline and online and we can pass users' location to the nearest medical facilities. At last, a multilingual environment is also supported.
==Requirements==
The major requirements are that:
*Secure login;
*Multiple languages;
*users are able to work offline, and update forms and content when online;
*The initial forms that will be implemented are Hospitals, and Shelters;

==Architecture==

Below is the overview of the Eden Mobile Platform. It uses a Model-View-Controller design pattern as the foundation of the project. And it also introduces a "plugin" mechanism for adding new features.

[[File:S1503_Architecture.png‎ |frame|center|Use Case Diagram]]

==Use Cases==

* '''1. Security Log in'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills login form, which includes username, password and Captcha code.
*** Submit form to Sahana Server. 
* '''2. Submit forms to server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to Sahana Server. 
* '''3. Submit forms to local storage'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to local database. 
* '''4. Download new forms from server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Download new forms from server 
* '''5. Synchronize local data with server '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Synchronize local data with server 
* '''6. Multi-Language Support '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Choose a language from settings page 
* The UML use case diagram is shown below:

[[File:S1503_Usecase.png‎ |frame|center|Use Case Diagram]]

=Implementation=
==Add Multilanguage Support==
===Development-before preparation===
This program is hosted by a very experienced software engineering, who set the blueprint of this whole program. So we communicated with him serval times to get a better understanding of the project need and expectation. Also, he introduced the framework he used and showed some little tricky to us on google group discussion. Then we understand a mapping relationship and the implementation details about how to build a multilanguage support.

[[File:mapping.png]]

===Code Changes===
The html button changes in languages.js:

[[File:languagesjs.png]]

The html button changes in languagesController.js:

[[File:languagecontroller.png]]

Code changes in index.js:

[[File:index.png]]

Our basic implementation translation in controller.js function goes like this:

[[File:basic.png]]

languages.xml mapping information:

[[File:xml.png]]

===Multilanguage UI===
After serval times of discussion and communication, we finally realize the following UI. You can click the language you need to change to it. It's very easy-using and user comfortable. The following screenshot is what it looks like.

After clicking the English button, the English UI shows as following:

[[File:English.png]]

After clicking the Chinese button, the Chinese UI shows as following:

[[File:Chinese.png]]

After clicking the French button, the French UI shows as following:

[[File:French.png]]

After clicking the Bosnian button, the Bosnian UI shows as following:

[[File:Bosnian.png]]

=Troubleshooting=

1. When loading Sahana Eden Mobile's main page, browser console will print the following error message:

[[File:S1503_Error1.png‎ |frame|center|underscore-min.map not found error]]

The problem can be solved by downloading the file [https://github.com/jashkenas/underscore/blob/master/underscore-min.map underscore-min.map] and put it in the directory where underscore-min.js is stored. More information about this error can be found [http://stackoverflow.com/questions/18429625/missing-javascript-map-file-for-underscore-js-when-loading-asp-net-web-page here].

2. When adding multi-language module to Sahana Eden Mobile's main page, browser console will print the following error message:
After adding a language plugin module, users can see a new button called "Language", but it cannot be clicked and what makes worse is that the module affect other modules, for example, we cannot new a shelter. The framework is super complex and we are new here. After discussion, we write a email describing our problems to Tom Baker, the host developer, for help.

[[File:problem.png]]

After we received his short reply, we tried to follow his instruction to check some files but still get no idea what's our problem.

[[File:reply.png]]

Finally, we set breakpoints and used chrome's developer tool to help us debug. Fortunately, we found the an object is NULL, which affect the initiation. Then we found that in our controller there is a little error which leaded to this problem. After revised that, it run well now.

=References=
<references/>

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-23T19:09:04Z

Ylu31: /* Architecture */

<big>S1503. Sahana Eden Mobile Client</big>

=Introduction=
==Purpose==
This project aims at adding new functions to the EdenMobile client app to serve users better.These functions will allow mobile users to add new shelters and hospitals, get information of hospitals and shelters offline/online, get GPS location of device, and provide multilingual support.

==Background==
===What’s Sahana Eden?===
Sahana Eden is an Open Source Humanitarian Platform which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is the latest evolution of Sahana Software which was initially developed by members of the information technology (IT) community in Sri Lanka to provide solutions for the relief effort following the 2004 Indian Ocean Tsunami.
<ref>[http://eden.sahanafoundation.org/ Sahana Eden Instroduction]</ref> 

===What does Sahana Eden do?===
Sahana Eden provides various of functions which are sorted in several different modules. 

'''Organization Registry''' 
*Sahana Eden’s Organization Registry can track what organizations do and organizations also could record their Offices, Warehouse and Field Sites including their locations so they can be mapped as well as links to other modules such as Human Resources, Assets and Inventory. 
'''Project Tracking''' 
*Sahana Eden provides a valuable tool which could tell people what are going on about the project to help organizations responding to disasters know where the greatest needs are and coordinate with others who are engaged in similar work. 
'''Human Resources''' 
*There are contact lists in Sahana Eden which can be access by users to ensure that the right people can be contacted at the right time. 
'''Inventory''' 
*Sahana Eden can be used to manage inventories of items and match requests for items with warehouses and other facilities which have them available. Sahana Eden can support multiple Catalogs of Items as well as providing alternative items to ensure more effective use of supplies. 
'''Assets''' 
*Sahana Eden is able to manage assets which are needed to respond to disasters, track where they are, who they have been assigned to, and what condition they are in. This ensures that assets are used effectively and efficiently. 
'''Assessments''' 
*Sahana Eden can be used to collect and analyze information from assessments to help organizations more effectively plan their disaster management activities. 
'''Scenarios & Events''' 
*To help organizations better plan for disasters, Sahana Eden can be used to plan for different scenarios, including recording what human resources, assets, facilities and tasks will be needed to effectively respond. 
'''Map''' 
*Sahana Eden has fully integrated mapping functionality which allows any location-based data to be visualized on a map. Maps provide situational awareness which is essential when either planning to prepare for or respond to a disaster. 
'''Shelter Management''' 
*When disasters are widespread and result in population displacement, understanding and tracking the landscape of shelters - and the people in them - is a critical activity. The Shelter Registry provides functionality to list and track information on shelters and on the people arriving and departing. 
'''Messaging''' 
*Sahana Eden provides support for messages to be sent to one or Distribution Groups by Email, SMS, Twitter and Google Talk. Interactive messages can also be set up to allow people to send short message queries to Sahana Eden and receive automatic responses. 

===Sahana Eden Architecture===
The basic Sahana Eden architecture is as follows:

<table style="width: 100%; border: 1px dotted rgb(0, 0, 0); border-collapse: collapse;" cellpadding="5"><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Server</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Apache</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Other web servers can also be used, such as Cherokee.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Application</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Sahana Eden</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Application Framework</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Web2Py</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Programming Language</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Python & Java Script</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Database</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, or SQLite</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, and SQLite are supported.Other databases should be usable without major additional work since Web2Py supplies many connectors.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Operating System</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Linux (Debian recommended)</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Windows and Mac OS X are possible, but only recommended for single-user environments.</td>
</tr></table>

==Scope==
We will realize the signup, secure login and logout function, which would add new module into the following files:
*communication.js
*controller.js
*model.js
*view.js
We will make it available in multiple languages environment which will implement the languages fold.
We will add some new items including:“ hospital”, “vehicles” which will be added into the folder named “plugins”. The following file will be added into their corrsponding folders: 
Hospitals:
*mHospital.js (model)
*hospital.js, editHospital.js, hospital.html, hospital.css, editHospital.html (views)
*hospitalController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Vehicles:
*mVehicles.js (model)
*vehicles.js, editVehicles.js, vehicles.html, vehicles.css, editVehicles.html (views)
*vehiclesController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Also, we find some bugs and eliminate them. These could be any possible file in this project.

==Documentation==
former developer discussion about Sahana Oden
*An [https://docs.google.com/a/ncsu.edu/document/d/11bblC7klisYjN7Gjw6hDcs5TZ8uesJD3o2xnM2gI914/edit# overview of all the Sahana Eden projects]
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework basic introduction] about the framework of Edenmobile including application design, source<ref>[http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework Edenmobile Introduction]</ref>
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile#J2ME list] of all the requirements need to be done in this project.
*An [https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/sahana-eden google discussion], explaining former problems developers met and corresponding solutions
*[https://github.com/szhao8/eden Open source code about Sahana Eden]<ref>[https://github.com/szhao8/eden Sahana Eden Github]</ref>
*[https://github.com/szhao8/EdenMobile Open source code about Eden Mobile]<ref>[https://github.com/szhao8/EdenMobile EdenMobile Github]</ref>

=Design=
==Overview==
In this design, we add several modules to the EdenMobile client app to allow mobile users to add new shelters and hospitals. Also, users can see the list of hospitals and shelters both offline and online and we can pass users' location to the nearest medical facilities. At last, a multilingual environment is also supported.
==Requirements==
The major requirements are that:
*Secure login;
*Multiple languages;
*users are able to work offline, and update forms and content when online;
*The initial forms that will be implemented are Hospitals, and Shelters;

==Architecture==

Below is the overview of the Eden Mobile Platform. It uses a Model-View-Controller design pattern as the foundation of the project. And it also introduces a "plugin" mechanism for adding new features.
[[File:architecture.jpg]]

==Use Cases==

* '''1. Security Log in'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills login form, which includes username, password and Captcha code.
*** Submit form to Sahana Server. 
* '''2. Submit forms to server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to Sahana Server. 
* '''3. Submit forms to local storage'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to local database. 
* '''4. Download new forms from server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Download new forms from server 
* '''5. Synchronize local data with server '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Synchronize local data with server 
* '''6. Multi-Language Support '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Choose a language from settings page 
* The UML use case diagram is shown below:

[[File:S1503_Usecase.png‎ |frame|center|Use Case Diagram]]

=Implementation=
==Add Multilanguage Support==
===Development-before preparation===
This program is hosted by a very experienced software engineering, who set the blueprint of this whole program. So we communicated with him serval times to get a better understanding of the project need and expectation. Also, he introduced the framework he used and showed some little tricky to us on google group discussion. Then we understand a mapping relationship and the implementation details about how to build a multilanguage support.

[[File:mapping.png]]

===Code Changes===
The html button changes in languages.js:

[[File:languagesjs.png]]

The html button changes in languagesController.js:

[[File:languagecontroller.png]]

Code changes in index.js:

[[File:index.png]]

Our basic implementation translation in controller.js function goes like this:

[[File:basic.png]]

languages.xml mapping information:

[[File:xml.png]]

===Multilanguage UI===
After serval times of discussion and communication, we finally realize the following UI. You can click the language you need to change to it. It's very easy-using and user comfortable. The following screenshot is what it looks like.

After clicking the English button, the English UI shows as following:

[[File:English.png]]

After clicking the Chinese button, the Chinese UI shows as following:

[[File:Chinese.png]]

After clicking the French button, the French UI shows as following:

[[File:French.png]]

After clicking the Bosnian button, the Bosnian UI shows as following:

[[File:Bosnian.png]]

=Troubleshooting=

1. When loading Sahana Eden Mobile's main page, browser console will print the following error message:

[[File:S1503_Error1.png‎ |frame|center|underscore-min.map not found error]]

The problem can be solved by downloading the file [https://github.com/jashkenas/underscore/blob/master/underscore-min.map underscore-min.map] and put it in the directory where underscore-min.js is stored. More information about this error can be found [http://stackoverflow.com/questions/18429625/missing-javascript-map-file-for-underscore-js-when-loading-asp-net-web-page here].

2. When adding multi-language module to Sahana Eden Mobile's main page, browser console will print the following error message:
After adding a language plugin module, users can see a new button called "Language", but it cannot be clicked and what makes worse is that the module affect other modules, for example, we cannot new a shelter. The framework is super complex and we are new here. After discussion, we write a email describing our problems to Tom Baker, the host developer, for help.

[[File:problem.png]]

After we received his short reply, we tried to follow his instruction to check some files but still get no idea what's our problem.

[[File:reply.png]]

Finally, we set breakpoints and used chrome's developer tool to help us debug. Fortunately, we found the an object is NULL, which affect the initiation. Then we found that in our controller there is a little error which leaded to this problem. After revised that, it run well now.

=References=
<references/>

File:S1503 Error1.png

2015-04-13T00:34:08Z

Ylu31:

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-13T00:33:56Z

Ylu31:

<big>S1503. Sahana Eden Mobile Client</big>

=Introduction=
==Purpose==
This project aims at adding new functions to the EdenMobile client app to serve users better.These functions will allow mobile users to add new shelters and hospitals, get information of hospitals and shelters offline/online, get GPS location of device, and provide multilingual support.

==Background==
===What’s Sahana Eden?===
Sahana Eden is an Open Source Humanitarian Platform which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is the latest evolution of Sahana Software which was initially developed by members of the information technology (IT) community in Sri Lanka to provide solutions for the relief effort following the 2004 Indian Ocean Tsunami.
<ref>[http://eden.sahanafoundation.org/ Sahana Eden Instroduction]</ref> 

===What does Sahana Eden do?===
Sahana Eden provides various of functions which are sorted in several different modules. 

'''Organization Registry''' 
*Sahana Eden’s Organization Registry can track what organizations do and organizations also could record their Offices, Warehouse and Field Sites including their locations so they can be mapped as well as links to other modules such as Human Resources, Assets and Inventory. 
'''Project Tracking''' 
*Sahana Eden provides a valuable tool which could tell people what are going on about the project to help organizations responding to disasters know where the greatest needs are and coordinate with others who are engaged in similar work. 
'''Human Resources''' 
*There are contact lists in Sahana Eden which can be access by users to ensure that the right people can be contacted at the right time. 
'''Inventory''' 
*Sahana Eden can be used to manage inventories of items and match requests for items with warehouses and other facilities which have them available. Sahana Eden can support multiple Catalogs of Items as well as providing alternative items to ensure more effective use of supplies. 
'''Assets''' 
*Sahana Eden is able to manage assets which are needed to respond to disasters, track where they are, who they have been assigned to, and what condition they are in. This ensures that assets are used effectively and efficiently. 
'''Assessments''' 
*Sahana Eden can be used to collect and analyze information from assessments to help organizations more effectively plan their disaster management activities. 
'''Scenarios & Events''' 
*To help organizations better plan for disasters, Sahana Eden can be used to plan for different scenarios, including recording what human resources, assets, facilities and tasks will be needed to effectively respond. 
'''Map''' 
*Sahana Eden has fully integrated mapping functionality which allows any location-based data to be visualized on a map. Maps provide situational awareness which is essential when either planning to prepare for or respond to a disaster. 
'''Shelter Management''' 
*When disasters are widespread and result in population displacement, understanding and tracking the landscape of shelters - and the people in them - is a critical activity. The Shelter Registry provides functionality to list and track information on shelters and on the people arriving and departing. 
'''Messaging''' 
*Sahana Eden provides support for messages to be sent to one or Distribution Groups by Email, SMS, Twitter and Google Talk. Interactive messages can also be set up to allow people to send short message queries to Sahana Eden and receive automatic responses. 

===Sahana Eden Architecture===
The basic Sahana Eden architecture is as follows:

<table style="width: 100%; border: 1px dotted rgb(0, 0, 0); border-collapse: collapse;" cellpadding="5"><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Server</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Apache</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Other web servers can also be used, such as Cherokee.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Application</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Sahana Eden</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Application Framework</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Web2Py</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Programming Language</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Python & Java Script</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Database</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, or SQLite</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, and SQLite are supported.Other databases should be usable without major additional work since Web2Py supplies many connectors.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Operating System</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Linux (Debian recommended)</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Windows and Mac OS X are possible, but only recommended for single-user environments.</td>
</tr></table>

==Scope==
We will realize the signup, secure login and logout function, which would add new module into the following files:
*communication.js
*controller.js
*model.js
*view.js
We will make it available in multiple languages environment which will implement the languages fold.
We will add some new items including:“ hospital”, “vehicles” which will be added into the folder named “plugins”. The following file will be added into their corrsponding folders: 
Hospitals:
*mHospital.js (model)
*hospital.js, editHospital.js, hospital.html, hospital.css, editHospital.html (views)
*hospitalController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Vehicles:
*mVehicles.js (model)
*vehicles.js, editVehicles.js, vehicles.html, vehicles.css, editVehicles.html (views)
*vehiclesController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Also, we find some bugs and eliminate them. These could be any possible file in this project.

==Documentation==
former developer discussion about Sahana Oden
*An [https://docs.google.com/a/ncsu.edu/document/d/11bblC7klisYjN7Gjw6hDcs5TZ8uesJD3o2xnM2gI914/edit# overview of all the Sahana Eden projects]
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework basic introduction] about the framework of Edenmobile including application design, source<ref>[http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework Edenmobile Introduction]</ref>
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile#J2ME list] of all the requirements need to be done in this project.
*An [https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/sahana-eden google discussion], explaining former problems developers met and corresponding solutions
*[https://github.com/szhao8/eden Open source code about Sahana Eden]<ref>[https://github.com/szhao8/eden Sahana Eden Github]</ref>
*[https://github.com/szhao8/EdenMobile Open source code about Eden Mobile]<ref>[https://github.com/szhao8/EdenMobile EdenMobile Github]</ref>

=Design=
==Overview==
In this design, we add several modules to the EdenMobile client app to allow mobile users to add new shelters and hospitals. Also, users can see the list of hospitals and shelters both offline and online and we can pass users' location to the nearest medical facilities. At last, a multilingual environment is also supported.
==Requirements==
The major requirements are that:
*Secure login;
*Multiple languages;
*users are able to work offline, and update forms and content when online;
*The initial forms that will be implemented are Hospitals, and Shelters;

==Architecture==
==Use Cases==

* '''1. Security Log in'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills login form, which includes username, password and Captcha code.
*** Submit form to Sahana Server. 
* '''2. Submit forms to server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to Sahana Server. 
* '''3. Submit forms to local storage'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to local database. 
* '''4. Download new forms from server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Download new forms from server 
* '''5. Synchronize local data with server '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Synchronize local data with server 
* '''6. Multi-Language Support '''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** Choose a language from settings page 
* The UML use case diagram is shown below:

[[File:S1503_Usecase.png‎ |frame|center|Use Case Diagram]]

=Troubleshooting=

1. When loading Sahana Eden Mobile's main page, browser console will print the following error message:

[[File:S1503_Error1.png‎ |frame|center|underscore-min.map not found error]]

The problem can be solved by downloading the file [https://github.com/jashkenas/underscore/blob/master/underscore-min.map underscore-min.map] and put it in the directory where underscore-min.js is stored. More information about this error can be found [http://stackoverflow.com/questions/18429625/missing-javascript-map-file-for-underscore-js-when-loading-asp-net-web-page here].

=References=
<references/>

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-02T03:49:39Z

Ylu31: /* Scope */

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-02T03:35:57Z

Ylu31: /* Purpose */

CSC/ECE 517 Spring 2015 S1503 LWJZ

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-01T00:59:56Z

Ylu31: /* Use Case */

<big>E1526. JavaRosa Mobile Client</big>
=Introduction=
==Purpose==
To add several modules to the EdenMobile client app based on the web2py framework to allow mobile users to add new shelters and hospitals, to see the list of hospitals and shelters both offline and online and pass their location to the nearest medical facilities.
==Scope==
We will realize the signup, secure login and logout function, which would add new module into the following files
*communication.js
*controller.js
*model.js
*view.js
We will make it available in multiple languages environment which will implement the languages fold.
We will add some new items including:“ hospital”, “vehicles” which will be added into the folder named “plugins”. The following file will be added into their corrsponding folders: 
Hospitals:
*mHospital.js (model)
*hospital.js, editHospital.js, hospital.html, hospital.css, editHospital.html (views)
*hospitalController.js (controller)
*config.js(Giving details on the configuration of the plugin
Vehicles:
*mVehicles.js (model)
*vehicles.js, editVehicles.js, vehicles.html, vehicles.css, editVehicles.html (views)
*vehiclesController.js (controller)
*config.js(Giving details on the configuration of the plugin)
Also, we find some bugs and eliminate them. These could be any possible file in this project.

==Background==
===What’s Sahana Eden?===
Sahana Eden is an Open Source Humanitarian Platform which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is the latest evolution of Sahana Software which was initially developed by members of the information technology (IT) community in Sri Lanka to provide solutions for the relief effort following the 2004 Indian Ocean Tsunami. 

===What does Sahana Eden do?===
Sahana Eden provides various of functions which are sorted in several different modules. 

===Organization Registry===
Sahana Eden’s Organization Registry can track what organizations do and organizations also could record their Offices, Warehouse and Field Sites including their locations so they can be mapped as well as links to other modules such as Human Resources, Assets and Inventory. 
====Project Tracking====
By telling you Who’s Doing What, Where, and When, Sahana Eden provides a valuable tool to help organizations responding to disasters know where the greatest needs are and coordinate with others who are engaged in similar work. 
====Human Resources====
Sahana Eden can also be used to provide a contact list to ensure that the right people can be contacted at the right time. 
====Inventory====
Sahana Eden can be used to manage inventories of items and match requests for items with warehouses and other facilities which have them available. Sahana Eden can support multiple Catalogs of Items as well as providing alternative items to ensure more effective use of supplies. 
====Assets====
Sahana Eden is able to manage assets which are needed to respond to disasters, track where they are, who they have been assigned to, and what condition they are in. This ensures that assets are used effectively and efficiently. 
====Assessments====
Sahana Eden can be used to collect and analyze information from assessments to help organizations more effectively plan their disaster management activities. 
====Scenarios & Events====
To help organizations better plan for disasters, Sahana Eden can be used to plan for different scenarios, including recording what human resources, assets, facilities and tasks will be needed to effectively respond. 
====Map ====
Sahana Eden has fully integrated mapping functionality which allows any location-based data to be visualized on a map. Maps provide situational awareness which is essential when either planning to prepare for or respond to a disaster. 
====Shelter Management====
When disasters are widespread and result in population displacement, understanding and tracking the landscape of shelters - and the people in them - is a critical activity. The Shelter Registry provides functionality to list and track information on shelters and on the people arriving and departing. 
====Messaging====
Sahana Eden provides support for messages to be sent to one or Distribution Groups by Email, SMS, Twitter and Google Talk. Interactive messages can also be set up to allow people to send short message queries to Sahana Eden and receive automatic responses. 
===Sahana Eden Architecture===
The basic Sahana Eden architecture is as follows:

<table style="width: 100%; border: 1px dotted rgb(0, 0, 0); border-collapse: collapse;" cellpadding="5"><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Server</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Apache</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Other web servers can also be used, such as Cherokee.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Application</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Sahana Eden</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Web Application Framework</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Web2Py</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Programming Language</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Python & Java Script</td>
<td style="border: 1px dotted rgb(0, 0, 0);"></td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Database</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, or SQLite</td>
<td style="border: 1px dotted rgb(0, 0, 0);">MySQL, PostgreSQL, and SQLite are supported.Other databases should be usable without major additional work since Web2Py supplies many connectors.</td>
</tr><tr><td style="border: 1px dotted rgb(0, 0, 0);">Operating System</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Linux (Debian recommended)</td>
<td style="border: 1px dotted rgb(0, 0, 0);">Windows and Mac OS X are possible, but only recommended for single-user environments.</td>
</tr></table>

==Documentation==
former developer discussion about Sahana Oden
* An [https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/sahana-eden google discussion], explaining former problems developers met and corresponding solutions
*An [http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework basic introduction about the framework of Edenmobile including application design, source ]
Source code
http://eden.sahanafoundation.org/wiki/BluePrint/Mobile/EdenMobileFramework

=Design=
=='''Use Case'''==

* '''Use Case #1: Security Log in'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills login form, which includes username, password and Captcha code.
*** Submit form to Sahana Server.

* '''Use Case #2: Submit forms to server'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User logs in to Sahana Server.
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to Sahana Server.

* '''Use Case #3: Submit forms to local storage'''
** '''Actor:''' Mobile App User
** '''Actions:'''
*** User fills forms of shelter/warehouse/disaster etc.
*** Submit form to local database.

* '''Use Case #4: Download new forms from server'''
** '''Actor:''' Student finish work for first round
** '''Actions:'''
*** User logs in to Sahana Server.
*** Download new forms from server

* '''Use Case #5: Synchronize local data with server '''
** '''Actor:''' Student not finish work for first round
** '''Actions:'''
*** User logs in to Sahana Server.
*** Synchronize local data with server

[[File:TO_BE_ADDED.png‎ |frame|center|Use Case Diagram]]

=Troubleshoots=

CSC/ECE 517 Spring 2015 S1503 LWJZ

2015-04-01T00:47:53Z

Ylu31: /* Use Cases */

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T15:35:46Z

Ylu31: /* Change Rails 2 syntax to Rails 4 */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [ VCL IP] (Log in as user2/password. Go to [ VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]
* [https://travis-ci.org/expertiza/expertiza/builds/54516405 Travis CI build passed]

=='''Todo List'''==
Related files:
* users_controller.rb
* participants_controller.rb
* user.rb
* users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==

==='''Method'''===

* The primary way we find unused method is right-click on each method's name, then choose "Find Usages", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

* Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

==='''Implementation'''===

* In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

* And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

* There's one method '''''salt_first''''' in '''''User.rb''''' cannot be deleted. Salt is a small chunk of random data to the password before it's hashed. The salt is then stored along with the hash in the database, and used to check potentially valid passwords<ref>[https://github.com/codahale/bcrypt-ruby Salt]</ref>. Here's the function:

<pre>
def salt_first?
true
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
* One main problem(bug) in Expertiza is that you can create one new user from the UI, but the second time you try to create one, you get “undefined ‘with_scope’” (see [https://github.com/expertiza/expertiza/issues/504 Issue 504 in Github]), as the following picture shows:

[[File:WithScopeError.PNG |frame|center|Delete confirm box]]

* When calling '''''@user.save''''' method, it will implicitly call the '''''ActiveRecord::Base#with_scope''''' method. However, this method is no longer supported in current Rails 4.1.5 version, which is used by Expertiza. We had searched a lot through the Internet, but find very limited useful information on this topic. It doesn't seem reasonable for '''''@user.save''''' still calling the '''''with_scope''''' method, since all versions if Rails, as well as ActiveRecord, are marked as the latest in '''''Gemfile.lock'''''.

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "users_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

From the code we can see that the search method needs four parameters:
* role: user can only search users below his role
* user_id: current user id
* letter: keyword in search
* search_by: search by user name, full name or email

Base on this, we implemented the search method in "user.rb":
<pre>
def self.search_users(role, user_id, letter, search_by)
if search_by == '1' #search by user name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '2' # search by full name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and fullname like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '3' # search by email
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and email like ?", role.get_available_roles, user_id, search_filter )
else #default used when clicking on letters
search_filter = letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
end
users
end
</pre>

And the original "paginate_list" method is modified as:
<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
@search_by = params[:search_by]

# search for corresponding users
users = User.search_users(role, user_id, letter, @search_by)

# paginate
if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = users.paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = users.page(params[:page]).per_page(paginate_options["#{@per_page}"])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implementation'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T15:35:05Z

Ylu31: /* Change Rails 2 syntax to Rails 4 */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [ VCL IP] (Log in as user2/password. Go to [ VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]
* [https://travis-ci.org/expertiza/expertiza/builds/54516405 Travis CI build passed]

=='''Todo List'''==
Related files:
* users_controller.rb
* participants_controller.rb
* user.rb
* users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==

==='''Method'''===

* The primary way we find unused method is right-click on each method's name, then choose "Find Usages", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

* Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

==='''Implementation'''===

* In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

* And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

* There's one method '''''salt_first''''' in '''''User.rb''''' cannot be deleted. Salt is a small chunk of random data to the password before it's hashed. The salt is then stored along with the hash in the database, and used to check potentially valid passwords<ref>[https://github.com/codahale/bcrypt-ruby Salt]</ref>. Here's the function:

<pre>
def salt_first?
true
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
* One main problem(bug) in Expertiza is that you can create one new user from the UI, but the second time you try to create one, you get “undefined ‘with_scope’” (see [https://github.com/expertiza/expertiza/issues/504 Issue 504 in Github]), as the following picture shows:

[[File:WithScopeError.PNG |frame|center|Delete confirm box]]]

* When calling '''''@user.save''''' method, it will implicitly call the '''''ActiveRecord::Base#with_scope''''' method. However, this method is no longer supported in current Rails 4.1.5 version, which is used by Expertiza. We had searched a lot through the Internet, but find very limited useful information on this topic. It doesn't seem reasonable for '''''@user.save''''' still calling the '''''with_scope''''' method, since all versions if Rails, as well as ActiveRecord, are marked as the latest in '''''Gemfile.lock'''''.

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "users_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

From the code we can see that the search method needs four parameters:
* role: user can only search users below his role
* user_id: current user id
* letter: keyword in search
* search_by: search by user name, full name or email

Base on this, we implemented the search method in "user.rb":
<pre>
def self.search_users(role, user_id, letter, search_by)
if search_by == '1' #search by user name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '2' # search by full name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and fullname like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '3' # search by email
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and email like ?", role.get_available_roles, user_id, search_filter )
else #default used when clicking on letters
search_filter = letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
end
users
end
</pre>

And the original "paginate_list" method is modified as:
<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
@search_by = params[:search_by]

# search for corresponding users
users = User.search_users(role, user_id, letter, @search_by)

# paginate
if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = users.paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = users.page(params[:page]).per_page(paginate_options["#{@per_page}"])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implementation'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

File:WithScopeError.PNG

2015-03-22T15:34:02Z

Ylu31:

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T15:33:09Z

Ylu31: /* Change Rails 2 syntax to Rails 4 */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [ VCL IP] (Log in as user2/password. Go to [ VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]
* [https://travis-ci.org/expertiza/expertiza/builds/54516405 Travis CI build passed]

=='''Todo List'''==
Related files:
* users_controller.rb
* participants_controller.rb
* user.rb
* users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==

==='''Method'''===

* The primary way we find unused method is right-click on each method's name, then choose "Find Usages", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

* Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

==='''Implementation'''===

* In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

* And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

* There's one method '''''salt_first''''' in '''''User.rb''''' cannot be deleted. Salt is a small chunk of random data to the password before it's hashed. The salt is then stored along with the hash in the database, and used to check potentially valid passwords<ref>[https://github.com/codahale/bcrypt-ruby Salt]</ref>. Here's the function:

<pre>
def salt_first?
true
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
* One main problem(bug) in Expertiza is that you can create one new user from the UI, but the second time you try to create one, you get “undefined ‘with_scope’” (see [https://github.com/expertiza/expertiza/issues/504 Issue 504 in Github]), as the following picture shows:

[[File:WithScopeError.PNG]]

* When calling '''''@user.save''''' method, it will implicitly call the '''''ActiveRecord::Base#with_scope''''' method. However, this method is no longer supported in current Rails 4.1.5 version, which is used by Expertiza. We had searched a lot through the Internet, but find very limited useful information on this topic. It doesn't seem reasonable for '''''@user.save''''' still calling the '''''with_scope''''' method, since all versions if Rails, as well as ActiveRecord, are marked as the latest in '''''Gemfile.lock'''''.

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "users_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

From the code we can see that the search method needs four parameters:
* role: user can only search users below his role
* user_id: current user id
* letter: keyword in search
* search_by: search by user name, full name or email

Base on this, we implemented the search method in "user.rb":
<pre>
def self.search_users(role, user_id, letter, search_by)
if search_by == '1' #search by user name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '2' # search by full name
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and fullname like ?", role.get_available_roles, user_id, search_filter )
elsif search_by == '3' # search by email
search_filter = '%' + letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and email like ?", role.get_available_roles, user_id, search_filter )
else #default used when clicking on letters
search_filter = letter + '%'
users = User.order('name').where( "(role_id in (?) or id = ?) and name like ?", role.get_available_roles, user_id, search_filter )
end
users
end
</pre>

And the original "paginate_list" method is modified as:
<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
@search_by = params[:search_by]

# search for corresponding users
users = User.search_users(role, user_id, letter, @search_by)

# paginate
if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = users.paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = users.page(params[:page]).per_page(paginate_options["#{@per_page}"])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implementation'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:56:22Z

Ylu31: /* Delete Uncalled Methods */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [http://152.46.16.178:3001/ VCL IP] (Log in as user2/password. Go to [http://152.46.16.178:3001/analytic/index VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]

=='''Todo List'''==
Related files: users_controller.rb, participants_controller.rb, user.rb, users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==
The primary way we find unused method is right-click on each method's name, then choose "Find Usages", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

However, there's one method in '''''User.rb''''' that has no usage found, but contributes to Expertiza function and thus cannot be deleted. Here's the function:

<pre>
def salt_first?
true
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
'''with_scope'''

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "user_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implement'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:53:02Z

Ylu31: /* Delete Uncalled Methods */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [http://152.46.16.178:3001/ VCL IP] (Log in as user2/password. Go to [http://152.46.16.178:3001/analytic/index VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]

=='''Todo List'''==
Related files: users_controller.rb, participants_controller.rb, user.rb, users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==
The primary way we find unused method is right-click on each method's name, then choose "Find Usage", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
'''with_scope'''

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "user_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implement'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:51:55Z

Ylu31: /* Delete Uncalled Methods */

E1506: Refactoring, testing and new features related to “users”

=='''Overview'''==
===Code refactoring===
''Refactoring'' is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior<ref>[http://refactoring.com/ Refactoring]</ref>.
Refactoring adds to the value of any program that has at least one of the following shortcomings<ref>[http://jexp.de/papers/refactoring/refactoring/node9.html#SECTION00330000000000000000 Benefits of Code Refactoring] Michael Hunger. Oct. 25, 2000</ref>:

* Programs that are hard to read are hard to modify;
* Programs that have duplicate logic are hard to modify;
* Programs that require additional behavior that requires you to change running code are hard to modify;
* Programs with complex conditional logic are hard to modify. 

===Rspec===
RSpec is a [http://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD) framework for the [http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby programming language], inspired by JBehave. It contains its own mocking framework that is fully integrated into the framework based upon JMock. The framework can be considered a [http://en.wikipedia.org/wiki/Domain-specific_language domain-specific language] (DSL) and resembles a natural language specification<ref>[http://en.wikipedia.org/wiki/RSpec#cite_note-origin-4g Rspec]</ref>.

===Object-oriented Design Principles===
Object-oriented programming (OOP) is a programming language model organized around objects rather than "actions" and data rather than logic<ref>[http://searchsoa.techtarget.com/definition/object-oriented-programming Margaret Rouse. ''object-oriented programming (OOP) definition''. Aug 3, 2008]</ref>. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. Though an Object oriented language provides us with highly useful and important programming concepts like '''Inheritance''', '''Polymorphism''', '''Abstraction''' and '''Encapsulation''' which definitely makes the code more efficient, it is equally important to have the knowledge of using them in the code.
''Object Oriented Design'' Principles are core of OOPS programming<ref>[http://javarevisited.blogspot.com/2012/03/10-object-oriented-design-principles.html#ixzz3Ha8L3cfz Javin Paul. ''Blogspot''. March 3, 2012]</ref>. It is important to know these design principles, to create clean and modular design. There are many design principles that help us to create clean and efficient code.

__TOC__

=='''Project Resources'''==
* [https://github.com/CSC517-Proj2-E1506/expertiza GitHub Repository]
* [http://152.46.16.178:3001/ VCL IP] (Log in as user2/password. Go to [http://152.46.16.178:3001/analytic/index VCL analytic page])
* [https://github.com/expertiza/expertiza/pull/505 GitHub Pull Request Link]

=='''Todo List'''==
Related files: users_controller.rb, participants_controller.rb, user.rb, users/show.html.erb
* Find the un-called methods if any and delete them. [done]
* Change the Rails 2 syntax to Rails 4 style.
* Refactor users_controller.rb
** Change the white space for the second harf of this file, starts at “def edit”. [done]
** Separate the “paginate_list” method into two methods. The search method should be in model and the paginating method should be in the controller. [done]
* New feature: delete users
** A user can be deleted if (s)he has not participated in an assignment. [done]
** If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?” [done]
** If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden.
*** rename (javascript calling update method in users_controller.rb) [done]
*** different users have different delete methods. [done]
** If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted. [done]
** Write tests (with Rspec) for this feature.[done]
* Testing feature: search for users
** In rails 4 branch, admins can search for the users with 1) users’ login names 2) users’ last or first names and 3) users’ emails. Please write tests (with Rspec) for this feature [done]

=='''Delete Uncalled Methods'''==
The primary way we find unused method is right-click on each method's name, then choose "Find Usage", as the following picture shows:
[[File:FindUsages.PNG |frame|center|Delete confirm box]]

Then the finding results will be displayed in the bottom, as the following picture shows:
[[File:FindResult.PNG |frame|center|Delete confirm box]]

In this way we identified the following method from '''''UserController.rb''''':

<pre>
def self.participants_in(assignment_id)
users = Array.new
participants = AssignmentParticipant.find_by_parent_id(assignment_id)
participants.each{
|participant|
users << User.find(participant.user_id)
}
end
</pre>

And the following methods from '''''User.rb'''''

<pre>
def assign_random_password
if self.password.blank?
self.password = self.random_password
end
end

def self.random_password(size=8)
random_pronouncable_password((size/2).round) + rand.to_s[2,3]
end

def get_author_name
return self.fullname
end
</pre>

=='''Change Rails 2 syntax to Rails 4'''==
'''with_scope'''

=='''Refactor users_controller.rb'''==
===='''Refactor the “paginate_list” method'''====

The "paginate_list" method is a function in "user_controller.rb". It will be called if you do search in Manage->Users page. It has two components: search for users, paginate the results.

However, the controller should not know how the information is retrieved. So we would like to refactor this method by seperating it into search method and paginating method. The search method is in model and the paginating method is still in controller.

The code below is the "paginate_list" method in official Expertiza:

<pre>
# For filtering the users list with proper search and pagination.
def paginate_list(role, user_id, letter)
paginate_options = {"1" => 25, "2" => 50, "3" => 100}

# If the above hash does not have a value for the key,
# it means that we need to show all the users on the page
#
# Just a point to remember, when we use pagination, the
# 'users' variable should be an object, not an array

#The type of condition for the search depends on what the user has selected from the search_by dropdown
condition = "(role_id in (?) or id = ?) and name like ?" #default used when clicking on letters
search_filter = letter + '%'
@search_by = params[:search_by]
if @search_by == '1' #search by user name
condition = "(role_id in (?) or id = ?) and name like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '2' # search by full name
condition = "(role_id in (?) or id = ?) and fullname like ?"
search_filter = '%' + letter + '%'
elsif @search_by == '3' # search by email
condition = "(role_id in (?) or id = ?) and email like ?"
search_filter = '%' + letter + '%'
end

if (paginate_options["#{@per_page}"].nil?) #displaying all - no pagination
users = User.order('name').where( [condition, role.get_available_roles, user_id, search_filter]).paginate(:page => params[:page], :per_page => User.count)
else #some pagination is active - use the per_page
users = User.page(params[:page]).order('name').per_page(paginate_options["#{@per_page}"]).where([condition, role.get_available_roles, user_id, search_filter])
end
users
end
</pre>

=='''New feature: delete users'''==

===='''Design'''====

At the very beginning, we decide to use "Cascading Delete" to delete users. Because there are many relationship between different users. They can be reviewer, reviewee, teaching assistant, and so on.If we have to delete an user, we have to not only delete the record in ''user'' table, but also other related tables.

So, we divide all users into two set, one is '''new users''' without any relationship, the other is the '''old user''' with some relationships. And we find that it is quite easy to achieve the functionality of deleting new users. When deleting old users, we find some problems. Because old users may be a reviewer before and score some assignments. If we delete some old users, the assignments' review scores will be a mess.

After discussing with professor, we decide to deprecate"Cascading Delete". And we use below algorithm to handle user deletion.

[[File:OSS0.png‎ |frame|center|Confirm box flow diagram]]

* A user can be deleted if (s)he has not participated in an assignment;
* If the user is participating in an assignment, the system will ask, “User is participating in k assignments. Delete as a participant in these assignment(s)?”;

[[File:OSS1.png‎ |frame|center|Delete confirm box]]

* If the user has submitted or reviewed in any of these assignments, the system will say the user cannot be deleted, but offer to rename the user account to <current_account_name>_hidden;

[[File:OSS2.png‎ |frame|center|Rename confirm box]]

* Rename (javascript calling update method in users_controller.rb);

[[File:OSS3.png‎ |frame|center|Rename success 2]]

* If the person trying to delete does not want to rename the account, the system will just say that the user can’t be deleted.

[[File:OSS4.png‎ |frame|center|Cannot delete]]

===='''Implementation'''====

We add a '''userDeleteConfirmBox.js''' file to implement this unique confirm box algorithm.

* First, we overwrite the Rails default confirm function.
$.rails.allowAction = function(link) {
console.log(link.attr('data-relationship'))
console.log(link.attr('data-username'))
if ((!link.attr('data-confirm')) || (link.attr('data-relationship') && link.attr('data-relationship') == 'false')) {
return true;
}
$.rails.showConfirmDialog(link);
return false;
};
$.rails.confirmed = function(link) {
link.removeAttr('data-confirm');
return link.trigger('click.rails');
};

* Then, we write code to achieve customed confirm box using our own algorithm.
$.rails.showConfirmDialog = function(link) {
var message = link.attr('data-confirm');
//confirmation box style
$(function() {
$(html1).modal();
$("#dialog-confirm1").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
//return $.rails.confirmed(link);
$( this ).dialog( "close" );
$(html2).modal();
$("#dialog-confirm2").dialog({
width: 340,
buttons: {
"Cancel": function() {
$( this ).dialog( "close" );
location.reload();
},
"Yes": function() {
$('#rename').click()
$( this ).dialog( "close" );
$(html3).modal();
$("#dialog-confirm3").dialog({
width: 340,
});
},
"No, delete any way!": function() {
$( this ).dialog( "close" );
$(html4).modal();
$("#dialog-confirm4").dialog({
width: 340,
buttons: {
"Close": function() {
$( this ).dialog( "close" );
location.reload();
}
}
});
}
}
});
}
}
});
});
return $('#dialog-confirm .confirm').on('click', function() {
return $.rails.confirmed(link);
});
};

* After that, we all the if condition in '''/users/show.html.erb''' file. So when deleting the old users, Expertiza uses the functionality and customed confirm box we define; when deleting the new users, Expertiza will use the default confirm box.
<% if @assignment_participant_num != 0 and (!@maps.nil? or @maps.length != 0)%>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User is participating in #{@assignment_participant_num} assignments. Delete as a participant in these assignment(s)?", :relationship => 'true', :username => @user.name}, :method => :delete, remote: true%>
<% else %>
<%= link_to 'Delete', {:action => 'destroy', :id => @user}, data:{:confirm => "User joins in #{@assignment_participant_num} assignment(s), but without participates in
any assignments. Delete as a participant in these assignment(s)?", :relationship => 'false'}, :method => :delete, remote: true%>
<% end %>

=='''Testing feature: search for users'''==

==='''Design'''===

We write some features test to test "search for users" functionality.

* Instructor searches users bylogin names
feature 'Method 1: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by login name' do
visit '/users/list'
fill_in 'letter', with: 'student'
find('#search_by').select 'Username'
click_button 'Search'
expect(page).to have_content("Student, Perfect")
expect(page).to have_content("pstudent@dev.null")
end
end

* Instructor searches users by first name or last name
feature 'Method2: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by last or first name' do
visit '/users/list'
fill_in 'letter', with: 'Bob'
find('#search_by').select 'Full name'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor searches users by email
feature 'Method3: Instructor search a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'by email' do
visit '/users/list'
fill_in 'letter', with: 'bdole@dev.null'
find('#search_by').select 'Email'
click_button 'Search'
expect(page).to have_content("Dole, Bob")
expect(page).to have_content("bdole@dev.null")
end
end

* Instructor deletes new users
feature 'Instructor delete a user' do
before(:all) do
instructor.save
student.save
log_in instructor.name, "password"
end
scenario 'which has no relationship' do
visit '/users/list'
#in order to show whole user list
fill_in 'letter', with: ''
find('#search_by').select 'Username'
click_button 'Search'
click_link 'student'
click_link 'Delete'
expect(page).to_not have_content("student")
expect(page).to have_content("instructor")
end
end

==='''Implement'''===
rspec spec/features/users_spec.rb

[[File:Rspec1.png‎ |frame|center|Rspec feature test pass]]

=='''References'''==
<references/>

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:44:35Z

Ylu31: /* Delete Uncalled Methods */

File:FindResult.PNG

2015-03-22T01:43:27Z

Ylu31: uploaded a new version of "File:FindResult.PNG"

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:42:35Z

Ylu31: /* Delete Uncalled Methods */

File:FindResult.PNG

2015-03-22T01:42:04Z

Ylu31:

CSC/ECE 517 Spring 2015/oss E1506 SYZ

2015-03-22T01:41:35Z

Ylu31: /* Delete Uncalled Methods */

File:FindUsages.PNG

2015-03-22T01:37:54Z

Ylu31:

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T04:00:47Z

Ylu31: /* Conclusion */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Examples'''=
==Template streaming==
Some basic knowledge about how web pages are rendered can be found [http://friendlybit.com/css/rendering-a-web-page-step-by-step/ here] and also [http://www.pathinteractive.com/blog/2014/07/rendering-a-webpage-with-google-webmaster-tools/ here]. 
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming''] module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref>Template Streaming, George Ogata https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method ''yield'' and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like ''new'' or ''edit'', it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won't need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module [http://api.rubyonrails.org/classes/ActionController/Live.html ''::Live''] was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ''::Live'' requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ''ActionController::Live''. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ''ActionController::Live''. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>Is it live? http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
[https://en.wikipedia.org/wiki/Server-sent_events Server-sent event (SSE)] is a technology where browser can automatically receive update data from a server via [https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol HTTP] connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of [https://en.wikipedia.org/wiki/HTML5 HTML5] by the [https://en.wikipedia.org/wiki/World_Wide_Web_Consortium W3C]. It can be used together with the Live API to achieve full-duplex communication.<ref>Server-sent events https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in [https://en.wikipedia.org/wiki/JSON JSON] format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

='''Conclusion'''=
The supporting for live streaming has deep impact on Rails. It not only makes Rails more competitive with Node.js, but also opens a gate for Rails to embrace lots of new areas and applications that it does not fits well before. The main advantage of Rails can be summarized as follows: 

1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself.
 
3.Continuously inform the user about the progress.
See example below<ref>Why Rails 4 Live Streaming is a big deal, Hongli Lai http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original [https://en.wikipedia.org/wiki/WEBrick WEBrick] server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>Installing puma on windows, Robert Kranz https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T03:46:40Z

Ylu31: /* Template streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Examples'''=
==Template streaming==
Some basic knowledge about how web pages are rendered can be found [http://friendlybit.com/css/rendering-a-web-page-step-by-step/ here] and also [http://www.pathinteractive.com/blog/2014/07/rendering-a-webpage-with-google-webmaster-tools/ here]. 
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming''] module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref>Template Streaming, George Ogata https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method ''yield'' and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like ''new'' or ''edit'', it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won't need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module [http://api.rubyonrails.org/classes/ActionController/Live.html ''::Live''] was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ''::Live'' requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ''ActionController::Live''. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ''ActionController::Live''. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>Is it live? http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
[https://en.wikipedia.org/wiki/Server-sent_events Server-sent event (SSE)] is a technology where browser can automatically receive update data from a server via [https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol HTTP] connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of [https://en.wikipedia.org/wiki/HTML5 HTML5] by the [https://en.wikipedia.org/wiki/World_Wide_Web_Consortium W3C]. It can be used together with the Live API to achieve full-duplex communication.<ref>Server-sent events https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in [https://en.wikipedia.org/wiki/JSON JSON] format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

='''Conclusion'''=
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>Why Rails 4 Live Streaming is a big deal, Hongli Lai http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original [https://en.wikipedia.org/wiki/WEBrick WEBrick] server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>Installing puma on windows, Robert Kranz https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T03:30:27Z

Ylu31: /* Template streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Some basic knowledge about how web pages are rendered can be found [http://friendlybit.com/css/rendering-a-web-page-step-by-step/ here] and [http://www.pathinteractive.com/blog/2014/07/rendering-a-webpage-with-google-webmaster-tools/ here]. 
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming''] module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref>Template Streaming, George Ogata https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method ''yield'' and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like ''new'' or ''edit'', it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won't need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module [http://api.rubyonrails.org/classes/ActionController/Live.html ''::Live''] was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ''::Live'' requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ''ActionController::Live''. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ''ActionController::Live''. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>Is it live? http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
[https://en.wikipedia.org/wiki/Server-sent_events Server-sent event (SSE)] is a technology where browser can automatically receive update data from a server via [https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol HTTP] connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of [https://en.wikipedia.org/wiki/HTML5 HTML5] by the [https://en.wikipedia.org/wiki/World_Wide_Web_Consortium W3C]. It can be used together with the Live API to achieve full-duplex communication.<ref>Server-sent events https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in [https://en.wikipedia.org/wiki/JSON JSON] format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>Why Rails 4 Live Streaming is a big deal, Hongli Lai http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original [https://en.wikipedia.org/wiki/WEBrick WEBrick] server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>Installing puma on windows, Robert Kranz https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T03:26:01Z

Ylu31: /* Template streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Some basic knowledge about how web pages are rendered can be found [http://friendlybit.com/css/rendering-a-web-page-step-by-step/ here] and [http://www.pathinteractive.com/blog/2014/07/rendering-a-webpage-with-google-webmaster-tools/ here]. 
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming''] module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref>Template Streaming, George Ogata https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won't need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>ActionController::Streaming http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module [http://api.rubyonrails.org/classes/ActionController/Live.html ''::Live''] was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ''::Live'' requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ''ActionController::Live''. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ''ActionController::Live''. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>Is it live? http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
[https://en.wikipedia.org/wiki/Server-sent_events Server-sent event (SSE)] is a technology where browser can automatically receive update data from a server via [https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol HTTP] connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of [https://en.wikipedia.org/wiki/HTML5 HTML5] by the [https://en.wikipedia.org/wiki/World_Wide_Web_Consortium W3C]. It can be used together with the Live API to achieve full-duplex communication.<ref>Server-sent events https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in [https://en.wikipedia.org/wiki/JSON JSON] format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>Why Rails 4 Live Streaming is a big deal, Hongli Lai http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original [https://en.wikipedia.org/wiki/WEBrick WEBrick] server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>Installing puma on windows, Robert Kranz https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

File:Nostreaming.gif

2015-02-16T03:10:13Z

Ylu31: uploaded a new version of "File:Nostreaming.gif"

No streaming example.

File:Nostreaming.gif

2015-02-16T03:06:16Z

Ylu31: uploaded a new version of "File:Nostreaming.gif"

No streaming example.

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:56:12Z

Ylu31: /* Template streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming''] module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won't need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module [http://api.rubyonrails.org/classes/ActionController/Live.html ''::Live''] was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ''::Live'' requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ''ActionController::Live''. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ''ActionController::Live''. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:54:27Z

Ylu31: /* Introduction */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

In Rails, the concept of streaming first appeared in [http://weblog.rubyonrails.org/releases/ version 3.1], and is further optimized in Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html ''::Streaming'' module] has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: '':Streaming'' only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield '':title'' in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ''::Live'' was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:38:19Z

Ylu31: /* Traditional Response */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

The concept of streaming first appeared in Rails 3.1, and is better further optimized along with the publish of Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html/ ''::Streaming'' module] has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: :Streaming only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield :title in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ''::Live'' was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows: 
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:37:52Z

Ylu31: /* Live Streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

The concept of streaming first appeared in Rails 3.1, and is better further optimized along with the publish of Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html/ ''::Streaming'' module] has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: :Streaming only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield :title in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ''::Live'' was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows:
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref> Like the image shows below: 
[[File:streaming.gif]]
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:36:35Z

Ylu31: /* Live Streaming */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

The concept of streaming first appeared in Rails 3.1, and is better further optimized along with the publish of Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, [http://api.rubyonrails.org/classes/ActionController/Streaming.html/ ''::Streaming'' module] has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ''::Streaming'' inverts the regular Rails process of rendering the layout and template. Using ''::Streaming'', Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: :Streaming only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield :title in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ''::Live'' was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows:
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now '''data can be streamed to the client immediately whenever write method is called'''. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 8 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref>Like the image shows below:
[[File:streaming.gif]]
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T02:34:46Z

Ylu31: /* Traditional Response */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

The concept of streaming first appeared in Rails 3.1, and is better further optimized along with the publish of Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, ::Streaming module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ::Streaming inverts the regular Rails process of rendering the layout and template. Using ::Streaming, Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: :Streaming only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield :title in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ::Live was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
8.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will '''keep loading and remain blank''' for 8 seconds, and shows all 8 lines of “This is a response message!” altogether, as with the following image shows:
[[File:nostreaming.gif]]

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
10.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now data can be streamed to the client each time when write method is called. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 10 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref>
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>

File:Streaming.gif

2015-02-16T02:33:01Z

Ylu31: Streaming example.

Streaming example.

File:Nostreaming.gif

2015-02-16T02:32:09Z

Ylu31: No streaming example.

No streaming example.

CSC/ECE 517 Spring 2015/ch1a 12 LS

2015-02-16T01:47:06Z

Ylu31: /* Introduction */

Live Streaming

='''Introduction'''=
The purpose of live streaming is to send data over the Internet in real-time. This requires a data source, a data encoder, a media publisher and a proper network.

The concept of streaming first appeared in Rails 3.1, and is better further optimized along with the publish of Rails 4. Now Rails 4 supports live streaming, which uses handling I/O object and can send data from server to client in real-time.

__TOC__

='''Details'''=
==Template streaming==
Since Rail 3.1, ::Streaming module has been added to ActionController. For example, the loading progress in a traditional Rails client-side may look like this<ref> https://github.com/oggy/template_streaming</ref>:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:slow-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 1 progress without streaming</div>
This is because that layouts are rendered before the content, thus causing control flow to be altered to render the page in the order that the client needs (layout first).But for some resources which are static, if they could be loaded while waiting for the server response, a lot of time can be saved.
 
 
The module ::Streaming inverts the regular Rails process of rendering the layout and template. Using ::Streaming, Rails will render template first and then layout. It will first run method yield and load up the template, and then renders the assets and layouts. In this way, the progress with streaming looks like this:
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">
[[File:fast-profile.png]]</div>
<div class="center" style="width: auto; margin-left: auto; margin-right: auto;">Figure 2 progress with streaming</div>

===When to use streaming===
For actions like new or edit, it may not be that necessary to use streaming. But for expensive actions like database query and generation of large amount of data, streaming can be extremely important and effective. Take the following action as an example:
<pre>
def storyboard
@stories = Story.all
@developers = Page.all
@comments = Comment.all
end
</pre>

Since most of the queries here are executing in the controller, it might take a while before them to finish, thus causing the front end (user) to wait for long. We can solve this problem and enhance user experience, the above code can be rewrite as follows,
<pre>
def storyboard
# Allow lazy execution of the queries
@stories = Story.all
@developers = Page.all
@comments = Comment.all
render :stream => true
end
</pre>
(Note: :Streaming only works with templates. It doesn’t work with :json or :xml)<ref>http://apidock.com/rails/ActionController/Streaming</ref>
 
 

===Tweaking views===
When using streaming, instance variables that are set in the template and are used in the layout won’t need some tweak to work well. This is done by using content_for, provide and yield.
Here’s a simple example where the layout expects the template to tell which title to use<ref>http://apidock.com/rails/ActionController/Streaming</ref>:
<pre>
<html>
<head><title><%= yield :title %></title></head>
<body><%= yield %></body>
</html>
</pre>

Instead of yield, we can also use content_for to specify the title:
<pre>
<%= content_for :title, "Main" %>
Hello
</pre>

And the final result would be:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>

However, if content_for is called several times, the final result would have all calls concatenated.
For instance, if we have the following template:
<pre>
<%= content_for :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>

The final result would be:
<pre>
<html>
<head><title>Main page</title></head>
<body>Hello</body>
</html>
</pre>
This means that, if you have yield :title in your layout and you want to use streaming, you would have to render the whole template (and eventually trigger all queries) before streaming the title and all assets, which kills the purpose of streaming. For this reason Rails 3.1 introduces a new helper called provide that does the same as content_for but tells the layout to stop searching for other entries and continue rendering.

For instance, the template above using provide would be:
<pre>
<%= provide :title, "Main" %>
Hello
<%= content_for :title, " page" %>
</pre>
Giving:
<pre>
<html>
<head><title>Main</title></head>
<body>Hello</body>
</html>
</pre>
That said, when streaming, you need to properly check your templates and choose when to use provide and content_for.
 
 

==Live Streaming==
The module ::Live was then added to ActionController at Rail 4. The difference between Streaming and Live is that Streaming is implicitly included whenever you create a new controller class, while ::Live requires explicit include.

===Traditional Response===
In this type of communication between server and client, data will be buffered until all render works are finished, and everything will be sent to the client at the same time. And the stream object behaves just like an IO object. Here’s the sample code:
<pre>
class MessageController < ApplicationController
def message
response.headers["Content-Type"] = "text/event-stream"
10.times {
response.stream.write "This is a response message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>

When we run the code above, the web page will remain blank for 10 seconds, and shows all 10 lines of “This is a response message!” altogether.

===Live Streaming===
In order to enable live streaming, we need to include a module named ActionController::Live. When this module is mixed into the controller, all actions within that controller will be able to stream data to the client in real time, like the following code:
<pre>
class MessageController < ApplicationController
#here we enabled live streaming
include ActionController::Live
def message
response.headers["Content-Type"] = "text/event-stream"
10.times {
response.stream.write "This is a live streaming message!\n"
sleep 1
}
ensure
response.stream.close
end
end
</pre>
The above code looks exactly the same as the traditional response, except the newly included module ActionController::Live. Now data can be streamed to the client each time when write method is called. When we run the code above, the web page will print one line “This is a response message!” for every seconds and repeats 10 times.<ref>http://tenderlovemaking.com/2012/07/30/is-it-live.html</ref>
 
 

==Server-Sent Event==
Server-sent event (SSE) is a technology where browser can automatically receive update data from a server via HTTP connection. Whenever server sends data, the SSE method in the browser will trigger an event.The SSE EventSource API is standardized as part of HTML5 by the W3C. It can be used together with the Live API to achieve full-duplex communication.<ref>https://en.wikipedia.org/wiki/Server-sent_events</ref>
 
A simple SSE looks like the following :
<pre>
require 'json'
module ServerSide
class SSE
def initialize io
@io = io
end

def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{object}nn"
end

def close
@io.close
end
end
end
</pre>
This module assigns the I/O stream object to a hash and converts it into a key-value pair so that it is easy to read, store, and send it back in JSON format.
 
 
Now we can wrap stream object inside the SSE class. By doing this, we need to include SSE module inside the controller, so that the opening and closing of connections will be managed by SSE module. And we also add the ensure clause to make sure that the connection will always be closed after execution.
<pre>
require 'sse'
class MessagingController < ApplicationController
include ActionController::Live

def stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
begin
5.times {
sse.write(“Streaming using SSE!”)
sleep 1
}
ensure
sse.close
end
end
end
</pre>
The code above will print the sentence “Streaming using SSE!” once per second, until the loop ends.
 
 
Here’s another sample showing how to use SSE to stream data<ref>https://github.com/rails/rails/blob/6061c540ac7880233a6e32de85cec72c20ed8778/actionpack/lib/action_controller/metal/live.rb</ref>:

# This class provides the ability to write an SSE (Server Sent Event)
# to an IO stream. The class is initialized with a stream and can be used
# to either write a JSON string or an object which can be converted to JSON.
#
# Writing an object will convert it into standard SSE format with whatever
# options you have configured. You may choose to set the following options:
#
# 1) Event. If specified, an event with this name will be dispatched on
# the browser.
# 2) Retry. The reconnection time in milliseconds used when attempting
# to send the event.
# 3) Id. If the connection dies while sending an SSE to the browser, then
# the server will receive a +Last-Event-ID+ header with value equal to +id+.
#
# After setting an option in the constructor of the SSE object, all future
# SSEs sent across the stream will use those options unless overridden.
#
# Example Usage:
#
# class MyController < ActionController::Base
# include ActionController::Live
#
# def index
# response.headers['Content-Type'] = 'text/event-stream'
# sse = SSE.new(response.stream, retry: 300, event: "event-name")
# sse.write({ name: 'John'})
# sse.write({ name: 'John'}, id: 10)
# sse.write({ name: 'John'}, id: 10, event: "other-event")
# sse.write({ name: 'John'}, id: 10, event: "other-event", retry: 500)
# ensure
# sse.close
# end
# end
#
# Note: SSEs are not currently supported by IE. However, they are supported
# by Chrome, Firefox, Opera, and Safari.

Note: By default Rails provides a one-way communication process by writing the stream to the client when data is available. However, if we can add SSEs, we can enable events and responses, thus making it two-way.
 
 

==Benefit of live streaming==
1.Send partial responses to the client immediately.
 
2.Using Server-sent events, the ability to create chat clients, push notifications, and real-time feeds are available within Rails itself
 
3.Continuously inform the user about the progress.
See example below<ref>http://blog.phusion.nl/2012/08/03/why-rails-4-live-streaming-is-a-big-deal/</ref>:
<pre>
def big_work
work = WorkModel.new
while !work.done?
work.do_some_calculations
response.stream.write "Progress: #{work.progress}%\n"
end
response.stream.close
end
</pre>
 

='''Troubleshooting'''=
When running streaming programs, we need to install [http://puma.io/ PUMA] to substitute the original WEBrick server, because puma can handle concurrency better. In order to do this, we need to add the following line to Gemfile:
<pre>
gem 'puma'
</pre>
 
After installed puma, if we start rails server, the console will print the following output:
<pre>
"C:\Program Files (x86)\JetBrains\RubyMine 7.0.2\bin\runnerw.exe" C:\RailsInstaller\Ruby2.0.0\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/Cloud/Coding/RubymineProjects/LiveStreamingExample/bin/rails server -b 127.0.0.1 -p 3000 -e development
=> Booting Puma
=> Rails 4.1.8 application starting in development on http://127.0.0.1:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Puma 2.11.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
</pre>
This means that PUMA is set as our default server and successfully booted.
 
 
We encountered failure on Windows when trying to install PUMA, the console printed the following error:
<pre>
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.

/Users/WillieTran/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
*** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
</pre>
 
To solve this problem, we need to install [https://www.openssl.org/ openssl] for windows. And then run the following command<ref>https://github.com/hicknhack-software/rails-disco/wiki/Installing-puma-on-windows</ref>:
<pre>
gem install puma -- --with-opt-dir=c:\openssl
</pre>
 

='''References'''=
<references>