CSC/ECE 517 Fall 2013/oss SocialMediaFeeds

Introduction to Eden

Sahana Eden is an Open Source Humanitarian Platform which has been built specifically for Disaster Management,Development, and Environmental Management sectors. It is highly configurable so that it can be used in a wide variety of different contexts and is easy to modify to build custom solutions.<ref>Tressel, Pat, Fran Boon, Shikhar Kohli, Dominic Koenig, Belinda Lopez, Eli Lev, Michael Howden, and Anne Goldenberg(2011, October) Sahana Eden Lulu.com, Retrieved from <http://en.flossmanuals.net/sahana-eden/index/> </ref> Sahana Eden can be accessed from the web or locally from a flash drive, allowing it to be used in environments with poor internet. Local & Web versions can be configured to synchronize to allow data to be shared between them. This helps to keep data synchronized to receive updates regularly and help in responding and recovering from disasters more effectively and efficiently.

Sahana Eden's modular design allows different pieces of functionality to be enabled and disabled as required providing flexible solutions for changing contexts.

The application can be configured to secure sensitive information, while also making data which needs to be shared available in a variety of different formats including Microsoft Excel and PDF. To ensure that Sahana Eden is accessible to every country, it can be translated into multiple languages.

Framework - Web2py

Web2py is an enterprise framework for agile development of secure database-driven web-based applications, written and programmable in Python.web2py is a full-stack framework, meaning that it contains all the components you need to build fully functional web applications.<ref> Massimo Di Pierro, Complete Reference Manual, 6th Edition (pre-release) Retrieved from <http://www.web2py.com/init/default/documentation> </ref>

Features of Web2py: <ref name = "name"> web2py.com </ref>

web2py includes a Database Abstraction Layer (DAL) that writes SQL dynamically so that you, the developer, don't have to. The DAL knows how to generate SQL transparently for SQLite, MySQL, PostgreSQL, MSSQL, FireBird, Oracle, IBM DB2, Informix and Ingres.

• The DAL can also generate function calls for the Google Datastore when running on the Google App Engine.
• web2py differs from other web frameworks in that it is the only framework to fully embrace the Web 2.0 paradigm, where the web is the computer.web2py does not require installation or configuration; it runs on any architecture that can run Python (Windows, Windows CE, Mac OS X, iOS, and Unix/Linux), and the development, deployment, and maintenance phases for the applications can be done via a local or remote web interface. web2py runs with CPython (the C implementation) and PyPy (Python written in Python), on Python versions 2.5, 2.6, and 2.7.
• web2py provides a ticketing system for error events. If an error occurs, a ticket is issued to the user, and the error is logged for the administrator.
• web2py is open source and released under the LGPL version 3 license.

Why Web2Py? <ref name="name"> Massimo Di Pierro, Complete Reference Manual, 6th Edition (pre-release) Retrieved from <http://web2py.com/books/default/chapter/29/01/introduction> </ref>

web2py is one of many web application frameworks, but it has compelling and unique features. web2py was originally developed as a teaching tool, with the following primary motivations:

• Easy for users to learn server-side web development without compromising functionality. For this reason, web2py requires no installation and no configuration, has no dependencies (except for the source code distribution, which requires Python 2.5 and its standard library modules), and exposes most of its functionality via a Web interface, including an Integrated Development Environment with Debugger and database interface.
• web2py has been stable from day one because it follows a top-down design; i.e., its API was designed before it was implemented. Even as new functionality has been added, web2py has never broken backwards compatibility, and it will not break compatibility when additional functionality is added in the future.
• web2py proactively addresses the most important security issues which plague many modern web applications, as determined by OWASP[owasp] below.
• web2py is lightweight. Its core libraries, including the Database Abstraction Layer, the template language, and all the helpers amount to 1.4MB. The entire source code including sample applications and images amounts to 10.4MB.
• web2py has a small footprint and is very fast. It uses the Rocket[rocket] WSGI web server developed by Timothy Farrell. It is as fast as Apache with mod_wsgi, and supports SSL and IPv6.
• web2py uses Python syntax for models, controllers, and views, but does not import models and controllers (as all the other Python frameworks do) - instead it executes them. This means that apps can be installed, uninstalled, and modified without having to restart the web server (even in production), and different apps can coexist without their modules interfering with one another.
• web2py uses a Database Abstraction Layer (DAL) instead of an Object Relational Mapper (ORM). From a conceptual point of view, this means that different database tables are mapped into different instances of one Table class and not into different classes, while records are mapped into instances of one Row class, not into instances of the corresponding table class. From a practical point of view, it means that SQL syntax maps almost one-to-one into DAL syntax, and there is no complex metaclass programming going on under the hood as in popular ORMs, which would add latency.

S3

The Sahana Eden framework (S3) is built on Web2Py and It is used for RAD(Papid Application Development) in all the sahana applications. Eden’s custom CRUD framework. S3 Framework includes pre configured Filters and Reports that allow the data analysis and report generation a simple task to be performed by directly.<ref>Tressel, Pat, Fran Boon, Shikhar Kohli, Dominic Koenig, Belinda Lopez, Eli Lev, Michael Howden, and Anne Goldenberg(2011, October) "Sahana Eden." Lulu.com, . Web. Oct. 2013. Retrieved from <http://en.flossmanuals.net/sahana-eden/technical-overview/> </ref>

S3 Filter forms allow the user to apply filters to interactive views (e.g. data tables, data lists, reports etc.). S3CRUD has filter forms integrated in the standard list view (=multi-record request w/o URL method). Filter forms can be rendered by any method handler. Since filter forms will never be submitted, the method handler does not need to handle any subsequent POSTs related to the filter form, but just insert the filter form HTML into the page.

S3 Framework also includes S3Report which is a REST method handler for data analysis. S3Report generates a full page which includes a set of S3Search widgets, a pivot table (a.k.a. contingency table) from S3Resources and graphs based on the Pivot Tables. S3Report responds to the /report URL method. S3Report supports a number of aggregation functions. The functions like list,count,sun and avg have been tested so far.

Table configurations can contain a report_options item, which is a Storage object. This object is used to configure reports and report options form. The rows, cols, facts and aggregate items are lists of available values for the user to select from. The defaults property is a Storage object that contains the default values for the report. It can contain a value for rows, cols, fact, aggregate and totals (as described in URL Methods). The search property is a list of S3Search widgets that will allow the report to be filtered. If no search property is specified, no filter form will be available.

Openshift Deployment

OpenShift is a cloud computing platform which offers "Platform As a Service" cloud. It is based on hosting platform provided by RedHat. It supports Ruby, it is really easy to install and start working on it. OpenShift takes care of maintaining the services underlying the application and scaling the application as needed. OpenShift supports web-application frameworks by supporting each language's preferred web-integration API, with no required changes to the actual framework code. web2py works unmodified on OpenShift.

Since the software is open source, it is available on GitHub. Development can be started after deploying different web applications in different languages on the platform using Git.Making Changes to your Application on OpenShift, you make code changes on your local machine, check those changes in locally, and then "push" those changes to OpenShift. One of the primary advantages of Git is that it does not require a continuous online presence in order to run. You can easily check in (in Git terminology, 'commit') and revert changes locally before deciding to upload those changes to OpenShift.

Configuration - Twitter

The Sahana Eden’s social media feeds requires the user to configure the twitter settings. This is done by providing the twitter login ID, and the OAuth credentials in the Twitter Settings page

Once the twitter settings are configured, we are all set to search twitter feeds.

Twitter Search After Save

Twitter search after save allows Sahana Eden to perform a search immediately after the search query is added.Search queries can be added in Sahana Eden by clicking on the Add query button in the Twitter Queries page.

The keyword to search is entered. Checking the "Search After Save?" will redirect the user to the results page once the search query is added.

Twitter Search Results Filter

The twitter search results for the search queries can be found in the Twitter Search Results page. The search filters on the Results page allow searching of feeds by "Tweeted date" and "Tweeted by".

Twitter Search Results Report

The Reports page generates Reports based on date filters and "tweeted by". The report contains the total number of records, search queries, Tweeted on and language that can further be grouped by search query, tweeted on or language.

Further Work (wrt to OSS Final Project)

The further work in the OSS project is listed below:

• Integration with Parsers
• rewire the 'parse_twitter' Parser
• Run Parsers on the list
• Extract Documents Parser
• Parser should run on any Message
• Parser should read attachments
• Parser should add all URLs as documents
• Parser should parse link types to see if there are Images
• optionally parser should be able to download linked images/documents into local store
• Train these parsers

Future Work

• The twitter search is currently hard coded. The search accesses the msg_twitter_search table and use the keywords stored by the user through the eden portal to search and display twitter feeds on the map.
• Display the twitter icon for twitter search instead of the usual bubble
• Work on twitter links and message which would pop up contents like photos,URL etc. This work is doing the part within the tweet parsing of following embedded URLs to identify if they're Media would be really useful addition to the parser (there is a commented-out stub).
• Create a cron job for searching tweets periodically instead of manual searches as per user’s request.
• Current implementation includes integration of Sahana with Twitter. The future enhancements of the eden project includes allow Sahana Eden to be configured to have a Facebook updates in it

Summary

The Sahana Eden Foundation developed for Disaster Management provides a web platform to provide solution for the relief effort. The project provides a solution to manage organizations, people, projects, inventory and assets as well as collecting information through assessments and providing situational awareness through maps. Hence, enhancing the searches and reports will provide great help to Disaster Management.

References

<references/>

Further Reading

1. Getting Started. Retrieved from https://dev.twitter.com/start
2. Sahana Eden Project. Retrieved from http://eden.sahanafoundation.org/wiki/WikiStart
3. Installation Guidelines. Retrieved from http://eden.sahanafoundation.org/wiki/InstallationGuidelines

Notes

• All references and further reading links are in the APA6 style
• All the content on this page is from websites that have a Attribution-NonCommercial-NoDerivs 3.0 or similar license that allows us to use their content.