Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T16:02:42Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very high speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
#[http://www.teradata.com/about-us/#tabbable=0&tab1=1 Tera Data]
#[http://www.vertica.com/ Vertica]

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. Please note that the above lists are not complete. For a novel web app, the tool might be out there, with no existing support for ruby or rails, and would require porting its libraries into a gem.

RoR community's support for the above mentioned tools is mature, but constantly evolving. The gems are available for all of them.

* For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]
* For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]
* For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

* For Tera Data [http://rubydoc.info/gems/activerecord-jdbcteradata-adapter/0.5.1/frames Tera data driver that enables the use of ActiveRecord with Tera Data]
* For Vertica [https://github.com/wvanbergen/vertica vertica gem]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014

2014-05-13T15:52:48Z

Sshah3:

*[[CSC/ECE_517_Fall_2012/example_page]]
*[[CSC/ECE 517 Spring 2014/ch1a 1a rt]]
*[[CSC/ECE 517 Spring 2014/ch1a 1i rt]]
*[[CSC/ECE 517 Spring 2014/ch1a 1e rm]]
*[[CSC/ECE 517 Spring 2014/ch1b 1j sr]]
*[[CSC/ECE 517 Spring 2014/ch1b 1o sr]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1h jg ]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1b np]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1f mj]]
*[[CSC/ECE 517 Spring 2014/ch1a 1d mm]]
*[[CSC/ECE 517 Spring 2014/ch1a 1c yj]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1l m]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1m bm]]
*[[CSC/ECE 517 Spring 2014/ch1a 1p fy]]
*[[CSC/ECE 517 Spring 2014/oss E1401 lmn]]
*[[CSC/ECE 517 Spring 2014/oss E1402 mmb]]
*[[CSC/ECE 517 Spring 2014/oss E1404 mnp]]
*[[CSC/ECE 517 Spring 2014/oss E1405 irw]]
*[[CSC/ECE 517 Spring 2014/oss E1406 st]]
*[[CSC/ECE 517 Spring 2014/oss S1402 jyy]]
*[[CSC/ECE 517 Spring 2014/oss E1413 mnp]]
*[[CSC/ECE 517 Spring 2014/oss E1414 st]]
*[[CSC/ECE 517 Spring 2014/security_audit]]
*[[CSC/ECE 517 Spring 2014/final_doc_updated qyx]]
*[[CSC/ECE 517 Spring 2014/E1412-Creating_Custom_Rubric-updated]]

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T15:17:30Z

Sshah3: /* Challenges */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very high speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
#[http://www.teradata.com/about-us/#tabbable=0&tab1=1 Tera Data]
#[http://www.vertica.com/ Vertica]

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. Please note that the above lists are not complete. For a novel web app, the tool might be out there, with no existing support for ruby or rails, and would require porting its libraries into a gem.

RoR community's support for the above mentioned tools is mature, but constantly evolving. The gems are available for all of them.

For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]
For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]
For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

For Tera Data [http://rubydoc.info/gems/activerecord-jdbcteradata-adapter/0.5.1/frames Tera data driver that enables the use of ActiveRecord with Tera Data]
For Vertica [https://github.com/wvanbergen/vertica vertica gem]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T15:01:10Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
#[http://www.teradata.com/about-us/#tabbable=0&tab1=1 Tera Data]
#[http://www.vertica.com/ Vertica]

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. Please note that the above lists are not complete. For a novel web app, the tool might be out there, with no existing support for ruby or rails, and would require porting its libraries into a gem.

RoR community's support for the above mentioned tools is mature, but constantly evolving. The gems are available for all of them.

For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]
For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]
For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

For Tera Data [http://rubydoc.info/gems/activerecord-jdbcteradata-adapter/0.5.1/frames Tera data driver that enables the use of ActiveRecord with Tera Data]
For Vertica [https://github.com/wvanbergen/vertica vertica gem]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:50:35Z

Sshah3: /* Important Terms */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
#[http://www.teradata.com/about-us/#tabbable=0&tab1=1 Tera Data]
#[http://www.vertica.com/ Vertica]

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. RoR community's support for the above mentioned tools is mature, but constantly evolving.

For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]

For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]

For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

For Tera Data [http://rubydoc.info/gems/activerecord-jdbcteradata-adapter/0.5.1/frames Tera data driver that enables the use of ActiveRecord with Tera Data]

For Vertica [https://github.com/wvanbergen/vertica vertica gem]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:50:11Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
#[http://www.teradata.com/about-us/#tabbable=0&tab1=1 Tera Data]
#[http://www.vertica.com/ Vertica]

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. RoR community's support for the above mentioned tools is mature, but constantly evolving.

For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]

For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]

For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

For Tera Data [http://rubydoc.info/gems/activerecord-jdbcteradata-adapter/0.5.1/frames Tera data driver that enables the use of ActiveRecord with Tera Data]

For Vertica [https://github.com/wvanbergen/vertica vertica gem]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:33:15Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#[http://cassandra.apache.org/ Cassandra]: An open source distributed DBMS designed to handle large amounts of data
#[http://www.mongodb.org/ MongoDB]: A distributed NoSQL database
#[http://couchdb.apache.org/ CouchDB]: Also a NoSQL databse that stores documents in JSON and uses javascript to make queries

'''Tools for data mining or Analytics'''
([Pivotal Greenplum Database]).

Using RoR to solve big data problems is a matter of identifying your particular use case, and then deciding on the right storage and analytic tools for that case. RoR community's support for the above mentioned tools is mature, but constantly evolving.

For MongoDB: [http://github.com/jnunemaker/mongomapper mondomapper]

For using CouchDB in a typical RoR RESTful way [https://github.com/couchrest/couchrest couchrest]

For Cassandra [https://github.com/cassandra-rb/cassandra cassandra]

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:14:21Z

Sshah3: /* Example */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#([http://cassandra.apache.org/ Cassandra]
#[http://www.mongodb.org/ MongoDB]
#[http://couchdb.apache.org/ CouchDB] )

'''Tools for data mining or Analytics'''
([Pivotal Greenplum Database]).
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:14:10Z

Sshah3:

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#([http://cassandra.apache.org/ Cassandra]
#[http://www.mongodb.org/ MongoDB]
#[http://couchdb.apache.org/ CouchDB] )

'''Tools for data mining or Analytics'''
([Pivotal Greenplum Database]).
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:13:30Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#([http://cassandra.apache.org/ Cassandra]
#[http://www.mongodb.org/ MongoDB]
#[http://couchdb.apache.org/ CouchDB] )

'''Tools for data mining or Analytics'''
([Pivotal Greenplum Database]).

==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T14:12:22Z

Sshah3: /* Big Data Usage in Rails Applications */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

Ruby is a dynamically-typed, interpreted programming language in the style of PHP or Python. Ruby on Rails is a framework to simplify the building of typical web apps, particularly CRUD apps. So neither of them in itself is a "big data" or "data mining" framework. However, for the purpose of storage or analytics, other tools are used, which do all the backend work, which is then then stored in the databases, which ruby can then access. Here we shall talk about RoR's compatibility with these different tools.

These tools can be broadly categorized into two categories:

'''Tools for Storage'''
#([http://cassandra.apache.org/ Cassandra]
##[http://www.mongodb.org/ MongoDB]
###[http://couchdb.apache.org/ CouchDB] )

'''Tools for data mining or Analytics'''
([Pivotal Greenplum Database]).

==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===

==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:37:16Z

Sshah3: /* Rails versus other frameworks for processing big data */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===
==Rails versus other frameworks for processing big data==
===Tools implemented on Python===
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:36:41Z

Sshah3: /* Tools implemented on Python */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===
===Rails versus other frameworks for processing big data===
====Tools implemented on Python====
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:35:59Z

Sshah3: /* Rails versus other frameworks for processing big data */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
===What is Big Data?===
[http://en.wikipedia.org/wiki/Big_data Big data] means a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity. <ref> http://www.webopedia.com/TERM/B/big_data.html </ref> The term may be used to refer the volumes of data, as well as the tools or techniques used to process, manage, analyze the data.

===Challenges===
As costs of storage are decreasing, it becomes trivial to store huge amounts of data, which leads to a much bigger challenge: determining relevance within volumes of data, and use analytics to create value from relevant data.

The real problem is not to acquire huge amounts of data, but how to make sense out of it to make any useful deduction. For example, if Google records and each and every search query that any of it's user makes, indexed with the Google account (where singed in) and IP address where not, the problem is not storage. The problem is predicting browsing habits, optimizing search results, creating profiles of google users, letting the profiles evolve with additional data, but only relevant data, deciding which data should be considered relevant etc.

Big data is often received at a very speed. We can consider the same example as mentioned above. Not only is data pouring in huge amounts, but it has to be structured, categorized, stored, managed at a very high speed .

Another challenge is that there's no set structure for big data (why would there be? big data is just data, just in huge volumes). For example, an international car parts supplier can index and store the sale of parts based on their make, model, item number, manfacturing data, location of purchase etc. This is a very structured form of big data, but as in our last example, where we only had a text string to determine all the variable, extraction of variables can be a challenge on unstructured big data.

Another challenge of big data, as with all emerging technologies is elastic scalability. If we're recording huge amounts of data, let's say views for a news website or shopping on a shopping portal, the data won't be consistent. There will be predictable (Christmas for the shopping website) and unpredictable (Any newsworthy outside event) occasions that will cause peaks and troughs in the inflow of data. Any system designed to store, manage, analyze big data should account for these as well.

==Example==
E-bay stores almost 90PB of data about customer transactions and behaviors to support some $3500 of product sales every second.

Data is stored in three systems, with about 7.5PB in a Teradata enterprise data warehouse, 40PB on commodity Hadoop clusters and 40PB on ‘Singularity’: a custom system for performing deep-dive analysis on semi-structured and relational data.

As of May 2013, eBay had 500 million live auction listings, split into more than 50,000 categories. The site has more than 100 million active users, generating up to 100TB of new data each day to be stored and used by more than 6000 eBay staff.

The users of this data, are of course company's key personnel whose job is to make important business decisions. They range from expert data scientists to non-technical business people and about 50 executives who need access to top-line reports.

E-bay has three separate teams just to manage this data. One team is responsible to look after all the technological details, and other to decide which kind of analytics are needed, considering the problems faced. <ref> http://www.itnews.com.au/News/342615,inside-ebay8217s-90pb-data-warehouse.aspx#ixzz31bJMJXRX </ref>

==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===

===Are there gems that facilitate it?===
===Rails versus other frameworks for processing big data===
==Tools implemented on Python==
Python has long been great for data munging and preparation, but less so for data analysis and modeling. [http://pandas.pydata.org/ pandas] helps fill this gap, enabling you to carry out your entire data analysis workflow in Python without having to switch to a more domain specific language like R.

Combined with the excellent [http://ipython.org/ IPython] toolkit and other libraries, the environment for doing data analysis in Python excels in performance, productivity, and the ability to collaborate.

[http://pandas.pydata.org/ pandas] does not implement significant modeling functionality outside of linear and panel regression; This functionality is provided by [http://ipython.org/ statsmodels] and [http://scikit-learn.org/stable/ scikit-learn]. <ref> http://pandas.pydata.org/ </ref>

Other tools based on python that cater to the needs of big data are [http://continuum.io/anaconda-server Anaconda] and [http://continuum.io/wakari Wakari]. Anaconda is a free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing. It has the libraries that facilitate running python scripts that can manage and analyse big data.
Wakari is a collaborative data analytics platform that includes tools to explore data, develop analytics scripts, collaborate with IPython notebooks, visualize, and share data analysis and findings. <ref> http://continuum.io/wakari </ref>

==Important Terms==
==References==
<references/>
* http://www.sas.com/en_us/insights/big-data/what-is-big-data.html

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:25:53Z

Sshah3: /* How easy it is to use Big Data in Rails Applications */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:21:04Z

Sshah3: /* Background */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:20:36Z

Sshah3: /* Usage */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:15:33Z

Sshah3: /* Examples */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:14:57Z

Sshah3: /* Examples */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T13:14:33Z

Sshah3: /* Examples */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T12:35:27Z

Sshah3: /* References */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T12:34:23Z

Sshah3: /* Background */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T12:11:08Z

Sshah3: /* Background */

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T10:51:23Z

Sshah3:

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
==Examples==
==Big Data Usage in Rails Applications==
===How easy it is to use Big Data in Rails Applications===
===Are there gems that facilitate it?===
===Rails versus other frameworks for processing big data===
==Important Terms==
==References==
<references/>

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T03:17:23Z

Sshah3: /* References */

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
==Examples==
==Important Terms==
==References==
<references/>

CSC/ECE 517 Spring 2014/ch1a 1o sr

2014-05-13T03:17:15Z

Sshah3:

This page covers the usage of Big Data with respect to Ruby on Rails.
==Background==
==Examples==
==Important Terms==
==References==

CSC/ECE 517 Spring 2014

2014-05-13T03:12:33Z

Sshah3:

*[[CSC/ECE_517_Fall_2012/example_page]]
*[[CSC/ECE 517 Spring 2014/ch1a 1a rt]]
*[[CSC/ECE 517 Spring 2014/ch1a 1i rt]]
*[[CSC/ECE 517 Spring 2014/ch1a 1e rm]]
*[[CSC/ECE 517 Spring 2014/ch1a 1j sr]]
*[[CSC/ECE 517 Spring 2014/ch1a 1o sr]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1h jg ]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1b np]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1f mj]]
*[[CSC/ECE 517 Spring 2014/ch1a 1d mm]]
*[[CSC/ECE 517 Spring 2014/ch1a 1c yj]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1l m]]
*[[CSC/ECE 517 Spring 2014/ch1 1w1m bm]]
*[[CSC/ECE 517 Spring 2014/ch1a 1p fy]]
*[[CSC/ECE 517 Spring 2014/oss E1401 lmn]]
*[[CSC/ECE 517 Spring 2014/oss E1402 mmb]]
*[[CSC/ECE 517 Spring 2014/oss E1404 mnp]]
*[[CSC/ECE 517 Spring 2014/oss E1405 irw]]
*[[CSC/ECE 517 Spring 2014/oss E1406 st]]
*[[CSC/ECE 517 Spring 2014/oss S1402 jyy]]
*[[CSC/ECE 517 Spring 2014/oss E1413 mnp]]
*[[CSC/ECE 517 Spring 2014/oss E1414 st]]
*[[CSC/ECE 517 Spring 2014/security_audit]]
*[[CSC/ECE 517 Spring 2014/final_doc_updated qyx]]
*[[CSC/ECE 517 Spring 2014/E1412-Creating_Custom_Rubric-updated]]

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:54:36Z

Sshah3:

==Background==
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

==Examples==

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
==Features==
The following two sections discuss Watir's features in relation to other frameworks that are used currently.

==Watir Compared to Capybara==
On their [http://watirmelon.com/tag/capybara/ blog], Watir team makes an interesting comparison of Capybara with Watir.

Basically, there are three ways, that is, three levels of details and control, that a programmer has to simulate a browser and the actions a user performs with an app in the browser. These three ways are [http://code.google.com/p/selenium/wiki/RubyBindings selenium-webdriver API],[http://watirwebdriver.com/ watir-webdriver API] and [http://rubydoc.info/github/jnicklas/capybara/master/Capybara/DSL Capybara DSL] <ref> http://watirmelon.com/tag/capybara/ </ref>.

Each level has it's pros and cons. While the lower level (Selenium) facilitate more control and flexibility, the higher level (Capybara) is convenient and quick to use and get up and running. Watir is somewhere in the middle.

The [http://watirmelon.com/tag/capybara/ author] uses the analogy of buying raw animal carcass, beef cuts, and prepared meats such as sausages to show the relationship between these three ways of interaction with the app.

[[File:threekindsofautomatedwebtestingapis.png]]

===Let's consider an example:===
Example taken from [http://watirmelon.com/tag/capybara/ this] page.
If we want to accomplish a fairly basic scenario on this example [http://bit.ly/watir-webdriver-demo Google Doc form]:

Start a browser
* Navigate to the watir-webdriver-demo form
* Check whether text field with id ‘entry_0′ exists (this should exist)
* Check whether text field with id ‘entry_99′ exists (this shouldn’t exist)
* Set a text field with id ‘entry_0′ to ’1′
* Set a text field with id ‘entry_0′ to ’2′
* Select ‘Ruby’ from select list with id ‘entry_1′
* Click the Submit button

<pre>
# * Start browser
# * Navigate to watir-webdriver-demo form
# * Check whether text field with id 'entry_0' exists
# * Check whether text field with id 'entry_99' exists
# * Set text field with id 'entry_0' to '1'
# * Set text field with id 'entry_0' to '2'
# * Select 'Ruby' from select list with id 'entry_1'
# * Click the Submit button

require 'bench'

benchmark 'selenium-webdriver' do
require 'selenium-webdriver'

driver = Selenium::WebDriver.for :firefox
driver.navigate.to 'http://bit.ly/watir-webdriver-demo'
begin
driver.find_element(:id, 'entry_0')
rescue Selenium::WebDriver::Error::NoSuchElementError
# doesn't exist
end
begin
driver.find_element(:id, 'entry_99').displayed?
rescue Selenium::WebDriver::Error::NoSuchElementError
# doesn't exist
end
driver.find_element(:id, 'entry_0').clear
driver.find_element(:id, 'entry_0').send_keys '1'
driver.find_element(:id, 'entry_0').clear
driver.find_element(:id, 'entry_0').send_keys '2'
driver.find_element(:id, 'entry_1').find_element(:tag_name => 'option', :value => 'Ruby').click
driver.find_element(:name, 'submit').click
driver.quit
end

benchmark 'watir-webdriver' do
require 'watir-webdriver'
b = Watir::Browser.start 'bit.ly/watir-webdriver-demo', :firefox
b.text_field(:id => 'entry_0').exists?
b.text_field(:id => 'entry_99').exists?
b.text_field(:id => 'entry_0').set '1'
b.text_field(:id => 'entry_0').set '2'
b.select_list(:id => 'entry_1').select 'Ruby'
b.button(:name => 'submit').click
b.close
end

benchmark 'capybara' do
require 'capybara'
session = Capybara::Session.new(:selenium)
session.visit('http://bit.ly/watir-webdriver-demo')
session.has_field?('entry_0') # => true
session.has_no_field?('entry_99') # => true
session.fill_in('entry_0', :with => '1')
session.fill_in('entry_0', :with => '2')
session.select('Ruby', :from => 'entry_1')
session.click_button 'Submit'
session.driver.quit
end

run 10

</pre>

As we can see from the above code that:

* Watir allows a little bit more neatness, for example, we don't have to write full URLs, complete with http://
* At the same time, it give enough power to specify the selectors explicitly. For example Capybara only supports name, id and label, but you can’t tell fill_in which specific one to choose: it appears to try each selector one by one until it finds it.

==Watir Compared to Cucumber==
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well. Consider the following example:

===Example===

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example below, the step definitions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

==Important Terms==

==References==
<references />
* http://watir.com/
* http://watir.com/examples/
* http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:51:13Z

Sshah3: /* Watir Compared to Capybara */

==Background==
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

==Examples==

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
==Features==

==Watir Compared to Capybara==
On their [http://watirmelon.com/tag/capybara/ blog], Watir team makes an interesting comparison of Capybara with Watir.

Basically, there are three ways, that is, three levels of details and control, that a programmer has to simulate a browser and the actions a user performs with an app in the browser. These three ways are [http://code.google.com/p/selenium/wiki/RubyBindings selenium-webdriver API],[http://watirwebdriver.com/ watir-webdriver API] and [http://rubydoc.info/github/jnicklas/capybara/master/Capybara/DSL Capybara DSL] <ref> http://watirmelon.com/tag/capybara/ </ref>.

Each level has it's pros and cons. While the lower level (Selenium) facilitate more control and flexibility, the higher level (Capybara) is convenient and quick to use and get up and running. Watir is somewhere in the middle.

The [http://watirmelon.com/tag/capybara/ author] uses the analogy of buying raw animal carcass, beef cuts, and prepared meats such as sausages to show the relationship between these three ways of interaction with the app.

[[File:threekindsofautomatedwebtestingapis.png]]

===Let's consider an example:===
Example taken from [http://watirmelon.com/tag/capybara/ this] page.
If we want to accomplish a fairly basic scenario on this example [http://bit.ly/watir-webdriver-demo Google Doc form]:

Start a browser
* Navigate to the watir-webdriver-demo form
* Check whether text field with id ‘entry_0′ exists (this should exist)
* Check whether text field with id ‘entry_99′ exists (this shouldn’t exist)
* Set a text field with id ‘entry_0′ to ’1′
* Set a text field with id ‘entry_0′ to ’2′
* Select ‘Ruby’ from select list with id ‘entry_1′
* Click the Submit button

<pre>
# * Start browser
# * Navigate to watir-webdriver-demo form
# * Check whether text field with id 'entry_0' exists
# * Check whether text field with id 'entry_99' exists
# * Set text field with id 'entry_0' to '1'
# * Set text field with id 'entry_0' to '2'
# * Select 'Ruby' from select list with id 'entry_1'
# * Click the Submit button

require 'bench'

benchmark 'selenium-webdriver' do
require 'selenium-webdriver'

driver = Selenium::WebDriver.for :firefox
driver.navigate.to 'http://bit.ly/watir-webdriver-demo'
begin
driver.find_element(:id, 'entry_0')
rescue Selenium::WebDriver::Error::NoSuchElementError
# doesn't exist
end
begin
driver.find_element(:id, 'entry_99').displayed?
rescue Selenium::WebDriver::Error::NoSuchElementError
# doesn't exist
end
driver.find_element(:id, 'entry_0').clear
driver.find_element(:id, 'entry_0').send_keys '1'
driver.find_element(:id, 'entry_0').clear
driver.find_element(:id, 'entry_0').send_keys '2'
driver.find_element(:id, 'entry_1').find_element(:tag_name => 'option', :value => 'Ruby').click
driver.find_element(:name, 'submit').click
driver.quit
end

benchmark 'watir-webdriver' do
require 'watir-webdriver'
b = Watir::Browser.start 'bit.ly/watir-webdriver-demo', :firefox
b.text_field(:id => 'entry_0').exists?
b.text_field(:id => 'entry_99').exists?
b.text_field(:id => 'entry_0').set '1'
b.text_field(:id => 'entry_0').set '2'
b.select_list(:id => 'entry_1').select 'Ruby'
b.button(:name => 'submit').click
b.close
end

benchmark 'capybara' do
require 'capybara'
session = Capybara::Session.new(:selenium)
session.visit('http://bit.ly/watir-webdriver-demo')
session.has_field?('entry_0') # => true
session.has_no_field?('entry_99') # => true
session.fill_in('entry_0', :with => '1')
session.fill_in('entry_0', :with => '2')
session.select('Ruby', :from => 'entry_1')
session.click_button 'Submit'
session.driver.quit
end

run 10

</pre>

As we can see from the above code that:

* Watir allows a little bit more neatness, for example, we don't have to write full URLs, complete with http://
* At the same time, it give enough power to specify the selectors explicitly. For example Capybara only supports name, id and label, but you can’t tell fill_in which specific one to choose: it appears to try each selector one by one until it finds it.

==Watir Compared to Cucumber==
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well. Consider the following example:

===Example===

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example below, the step definitions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

==Important Terms==

==References==
<references />
* http://watir.com/
* http://watir.com/examples/
* http://watirmelon.com/tag/capybara/

File:Threekindsofautomatedwebtestingapis.png

2014-05-08T12:40:46Z

Sshah3:

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:40:24Z

Sshah3: /* Watir Compared to Capybara */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:27:40Z

Sshah3: /* Examples/Usage */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:26:53Z

Sshah3: /* Writing Assignment 1j - Watir */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:24:34Z

Sshah3: /* Watir Compared to Cucumber */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this '''example''':

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
<references />
* http://watir.com/
* http://watir.com/examples/
* http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:24:00Z

Sshah3: /* References */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this example:

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
<references />
* http://watir.com/
* http://watir.com/examples/
* http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:20:43Z

Sshah3: /* References */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this example:

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
<references />
http://watir.com/
http://watir.com/examples/
http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:17:30Z

Sshah3: /* References */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this example:

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
http://watir.com/
http://watir.com/examples/
http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T12:17:08Z

Sshah3: /* References */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this example:

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
http://watir.com/

http://watir.com/examples/

http://watirmelon.com/tag/capybara/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T11:33:07Z

Sshah3: /* Watir Compared to Cucumber */

== Writing Assignment 1j - Watir ==

===Background===
Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
Watir is a family of Ruby libraries but it supports your app no matter what technology it is developed in. Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

===Example===

Here's Watir in action. Like all ruby gems, it has to be included to be used.
Including Watir gem to drive Internet Explorer on Windows
<pre>
require 'watir'
</pre>

Including Watir-WebDriver gem to drive Firefox/Chrome on Windows/Mac/Linux
<pre>
require 'watir-webdriver'
</pre>

After including it, It can be used simply. For example to start a new browser & and going to Watir's demo website:

<pre>
browser = Watir::Browser.new
browser.goto 'http://bit.ly/watir-example'
</pre>

For setting the value of a text field

[[File:Text-field.png]]
<pre>
browser.text_field(:name => 'entry.0.single').set 'Watir'
</pre>

For setting a multi-line text box

[[File:text-box1.png]]
<pre>
browser.text_field(:name => 'entry.1.single').set "I come here from Australia. \n The weather is great here."
</pre>

Their [http://watir.com/examples/ official example page] has a few examples.
===Features===

===Watir Compared to Capybara===

===Watir Compared to Cucumber===
Cucumber and Watir are not comparable in the sense that they are both different tools that aid each other for the same goal simple behavior-driven unit testing of the app. As demonstrated in the example, Watir is used to manipulate the browser, while Cucumber is used to write business readable unit tests. On its [http://cukes.info/ website], Cucumber mentions these six steps for suing Cucumber:

# Describe behaviour in plain text
# Write a step definition in Ruby
# Run and watch it fail
# Write code to make the step pass
# Run again and see the step pass
# Repeat 2-5 until green like a cuke

Basically "Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

In its [https://github.com/cucumber/cucumber/wiki/Browsers-and-Transactions community wiki] Cucumber mentions the usage of Watir (or Selenium, a similar tool) for making transactions with the browser. So these two tools go hand in hand in order to write a complete successful automated unit test suit, one that can be readable by non-technical people as well.

For instance, consider this example:

The way Cucumber works is that after installing the gem, the steps in business readable form are stored in .feature file, and then actual steps are written in the step definitions file. As you can see in the example belo, the step defintions can then use Watir to make transactions with the browser.

'''features/example.feature'''
<pre>
Feature: Search In order to use Google users must be able to search for content
Scenario: Search for a term
Given I have entered "watir" into the query
When I click "search"
Then I should see some results
</pre>
'''features/step_definitions/example_steps.rb'''
<pre>
require "watir-webdriver"
require "rspec/expectations"

Given /^I have entered "([^"]*)" into the query$/ do |term|
@browser ||= Watir::Browser.new :firefox
@browser.goto "google.com"
@browser.text_field(:name => "q").set term
end

When /^I click "([^"]*)"$/ do |button_name|
@browser.button.click
end

Then /^I should see some results$/ do
@browser.div(:id => "resultStats").wait_until_present
@browser.div(:id => "resultStats").should exist
@browser.close
end
</pre>

===Important Terms===

===References===
http://watir.com/
http://watir.com/examples/

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T11:27:18Z

Sshah3: /* Watir Compared to Cucumber */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T11:26:39Z

Sshah3: /* Watir Compared to Cucumber */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T11:25:04Z

Sshah3: /* Watir Compared to Cucumber */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T11:19:54Z

Sshah3: /* Watir Compared to Cucumber */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:54:53Z

Sshah3: /* Writing Assignment 1j - Watir */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:53:14Z

Sshah3: /* Example */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:52:36Z

Sshah3: /* Example */

File:Text-box1.png

2014-05-08T10:52:01Z

Sshah3:

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:51:30Z

Sshah3: /* Example */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:51:11Z

Sshah3: /* Example */

CSC/ECE 517 Spring 2014/ch1a 1j sr

2014-05-08T10:50:23Z

Sshah3: /* Example */