CSC/ECE 517 Fall 2014/final S1455: Difference between revisions

Latest revision as of 20:53, 16 December 2014

Please read the "README" file provided in expertiza. This wiki page contains design details for the project for Sahana Eden Humanitarian Platform, on their Central African Republic deployment.

Background Information

About Sahana Eden

Sahana Eden is an Open Source Humanitarian Platform, which can be used to provide solutions for Disaster Management, Development, and Environmental Management sectors. It is supported by the Sahana Software Foundation, Used in many Deployments, such as Ebola, Kashmir Floods, Hurricane Sandy, just to name a few. It is highly configurable so that it can be used in a wide variety of different contexts and is easy to modify to build custom solutions. Different levels of support are available from both the voluntary Sahana Eden community and professional companies. Sahana focuses on Community and Professional Support.

Getting to know the domain

Their domain page elaborately describes various major management projects and deployments that are done in the past or are ongoing. Spending some time to understand goals and purposes of Sahana before starting with actual development might help making the code better and more sensible from Sahana's perspective.

Installation & tutorials

Anyone who wishes to contribute their development skill to Sahana, can do so with the help of an elaborate guide they have provided, that covers everything from installation to tutorials on Python. Once the installations are complete, you can simply identify a task you are interested in, and start working on it. You may also want to sign their Contributor's License Agreement before making any pull request.

Below is a high level overview to begin contributing to Sahana-

The code is in the folder: web2py/applications/eden

Inside that folder are folders for Models (define the data structure), Controllers (provide URLs to enable access to the data) & Views (HTML templates).

Each module within Sahana Eden will normally consist of one of each of these files:

   Model: modules/s3db/modulename.py
   Controller: controllers/modulename.py
   View: views/modulename/index.html

In order to know which file to edit in order to change a particular function, you need to look at the URL. The Web2Py web framework maps URLs as follows:

   http://host/application/controller/function

So, if you want to edit the Home page with the URL:

   http://host/eden/default/index

This implies that you should look at the file eden/controllers/default.py and the index function within it which can be found by searching for the function title "def index():"

Tip: Sahana Eden makes heavy use of integrated resource controllers so the typical mapping is:

   http://host/eden/module/resource

The resource refers to a table with the name module_resource in the file modules/s3db/<module>.py

Introduction to project

Overview

In the Central African Republic page, some data needs new models, and some models need tweaking. The purpose of this project is to identify any such requirement and make necessary changes. The general steps to follow in Sahana development are mentioned below-

1. Add/modify the core models in modules/s3db

2. Add REST controllers for new models

3. Check basic operations using automatic CRUD interface- tweak list_fields to the most-relevant fields perhaps

4. Add menu entries (modules/s3menus.py)

5. Develop XSLT & CSV templates (static/formats/s3csv) to bulk import the data

6. Convert the data to these CSV templates- use ogr2ogr for this

7. Add layers to the map (private/templates/<TEMPLATE>/gis_layer_feature.csv. Look at default for now) including Markers and tooltips (popup_format)

8. Provide some report_options to make a meaningful report

Scope

Four modules related to this deployment are-

1. Health - modules/s3db/hms.py

2. Transport - modules/s3db/transport.py

3. Water - modules/s3db/water.py

4. School - There is no separate module for education.

We took up the Health module and worked on the Magnu template of Sahana. The key point for the core module is to ensure that we can capture all of the data that is available. There was a need to hide some of this complexity from the Magnu template which is used by CAR.

- Currently, Sahana is keen to show people how it could be used.

- An approach to this is to add one or two deployment_setting to be able to easily switch on/off chunks of functionality like this without needing to resort to custom code.

Another thing on a to-do list is adding markers to hospital overlay, and making them vary by type of hospital.

Reference Material

Hospital Model

Border Control Points, Roads, Routes, Airports

Water Bodies, Water Points, Rivers

Education

System Overview

Sahana Eden Framework:

The Sahana Eden Software Platform has been built around a Rapid Application Development (RAD) Framework. This provides a high level of automation to ensure that new solutions can be quickly and effectively developed. Once a database table is defined, the Sahana Eden Framework automatically generates HTML pages to handle CRUD (Create, Read, Update, Delete) as well as Search, Map and Pivot Reports. Web Services are available to import and export in XML, CSV, JSON and EXtensible Stylesheet Language (XSL) transforms are supported to produce other data standards.

The Sahana Eden Framework has flexible authorization policies which can be configured to grant permissions for different modules, tables as well as the ability to have multiple Organizations control their own data on a single Sahana Eden installation.

Sahana Eden can be downloaded and run locally from a flash drive. Synchronization functionality allows data to be entered then keep up to date between different installations, including online servers and local flash drive installations. The Sahana Eden Framework also includes a scheduler for running tasks at a specific time, in regular intervals or as asynchronous tasks which are triggered by users.

Sahana Eden is mainly implemented using Python, web2py, CSS and foundation. When you open the code in Eclipse, you can see that web2py is very much like Ruby on Rails. Analyze the screenshots below-

Our focus is on Controllers, Models, Modules and views. For every action on various pages on Sahana Humanitarian website, there are set of controllers defined. As mentioned above, we want to focus on 4 models- health, transport, water and school.

Sahana

Unicef Key	Unicef value (number)	OSM tag
all objects		`amenity=hospital`
all objects		`source=UNICEF`
ID		not tagged because not relevant
WPT		not tagged because not relevant
CODE		not tagged because not relevant
FOSA_CODE		not tagged because not relevant
LAT_DD		not tagged because not relevant
LONG_DD		not tagged because not relevant
PAYS		not tagged because not relevant
PREF, SOUS_PREF, COMM, VILLE_VILL, QUARTIER		`addrːfull=*`
VILLE_VILL		`addr:city=*`
POP_VILLE		not tagged because not relevant
STATUT_FOS	Public (306)	`operator:type=government`
STATUT_FOS	Prive (37)	`private`
TYPE_FOSA	Poste de sante (204)	`health_facility:type=dispensary`
TYPE_FOSA	Centre de sante (with beds) (118)	`health_facility:type=health_center`
TYPE_FOSA	Hopital prefectoral (7)	`health_facility:type=prefectoral_hospital`
TYPE_FOSA	Hopital regional (3)	`health_facility:typeregional_hospital`
TYPE_FOSA	Clinique privee (1)	`health_facility:type=private_hospital`
TYPE_FOSA	No attribut (1)	not to upload
NOM_FOSA		`name=*`
NB_LITS	numbers 0-179	`capacity:beds=*`
NB_PERS_SO		not tagged because unknown meaning
NB_PATIENT		not tagged because not relevant
COMITE_WAS	Oui/Non	not tagged because not relevant
NB_PERS_WA		not tagged because not relevant
LTA		not tagged because unknown meaning
VIP		not tagged because unknown meaning
VIP_PLEIN		not tagged because unknown meaning
LATRINES_S		not tagged because not relevant
LAVE_MAINS	Oui/Non	not tagged because not relevant
INCINERATE		not tagged because not relevant
BAC_A_ORDU	Oui/Non	not tagged because not relevant
TOTAL_LATR	numbers 0-23	`toilets:number=*`

System Architecture

The basic Sahana Eden architecture is as follows:

Web Server	Apache	Other web servers can also be used, such as Cherokee.
Application	Sahana Eden
Web Application Framework	Web2Py
Programming Language	Python & Java Script
Database	MySQL, PostgreSQL, or SQLite	MySQL, PostgreSQL, and SQLite are supported. Other databases should be usable without major additional work since Web2Py supplies many connectors.
Operating System	Linux (Debian recommended)	Windows and Mac OS X are possible, but only recommended for single-user environments.

Requirement Analysis

The requirements state that some of the data needs new models. Also, the school database states that there is no module for education. Even on the education website of Central African Republic, it says that the data is incomplete. Addition of new module, say edu.py in s3db module, will solve this problem.

For the purpose of this project, we will need to follow these steps-

1. Installation and setup- Follow the installation guidelines provided by Sahana to build the system on individual machines. Then, do the setup for Git repository, i.e. forking Sahana eden repo and adding the original as remote branch.

2. Identify properties for "edu.py" and define the basic data model in the models/ folder.

3. Add corresponding controller which provides access to this resource. Create another new file, this time in the controllers/ folder.

4. Define Field Types- By default fields are created with type string, however we may wish to use other data types. All fields have both client-side widgets & server-side validation automatically added based on their data type.

5. Customize and Internationalize field labels- Field labels are automatically generated from the field names, however we are able to customize these by adding a 'label' attribute.

6. Add Links to other Resources.

7. Override default CRUD Strings.

8. Update the Menus- There are two levels of menu within the system: The top-level 'Modules Menu' is visible in all modules. Underneath that, each module has its own menu for for module-specific navigation.

9. Test the changes.

Hiding the complexity

As you can see, the current hospital creation page shows many fields that are not required or necessary for CAR template. These fields need to be hidden from the user who wishes to add a hospital. Below figure shows the existing form (Left), and the actual form required (Right).

Doing this, requires following code changes-

GIS Requirements

Understanding OpenLayers & GeoExt

OpenLayers is an open source, client side JavaScript library for making interactive web maps, viewable in nearly any web browser. Since it is a client side library, it requires no special server side so beware or settings—you can use it without even downloading anything.Originally developed by Metacarta, as a response, in part, to Google Maps, it has grown into a mature, popular framework with many passionate developers and a very helpful community.

OpenLayers.Map

Here is an example that will create a page containing a full screen OpenLayers map. This is not the most basic of maps, but will work if you copy and paste the html content.

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="EN">
 <head>
  <style>
   html,body {
    height: 99%;
    width: 99%;
    }
   #map {
    width: 100%;
    height: 100%;
    border: 1px solid black;
    }	
   </style>
   <script src='http://openlayers.org/api/OpenLayers.js'>
   </script>
  </head>
  <body>
   < div id="map"></ div>
     <script type="text/javascript">
      var map = new OpenLayers.Map('map',{maxResolution: 0.703125} );
      var wmscURL = [
       "http://wmsc1.terrapages.net/getmap?",
       "http://wmsc2.terrapages.net/getmap?",
       "http://wmsc3.terrapages.net/getmap?",
       "http://wmsc4.terrapages.net/getmap?"
        ];
      var terrapagesStreetLayer = new OpenLayers.Layer.WMS( 'TerraPages Street',wmscURL, {layers: 'UnprojectedStreet', format: 'image/jpeg' }, {buffer: 1, isBaseLayer: true} );
      map.addLayer(terrapagesStreetLayer);
      map.zoomToMaxExtent();		
     </script>
   </body>
</html>

OpenLayers.Layer.Markers

Below is an example of how to create a Markers Layer, and add a single marker on that layer at the specified lon and lat. This will also apply an offest to the icon image, so that it can be aligned horizontally centered, vertically bottom over the position.

 var markersLayer = new OpenLayers.Layer.Markers("My Marker Layer");
 var iconSize =  new OpenLayers.Size(20,20);
 var iconOffset = new OpenLayers.Pixel(-(iconSize.w/2), -iconSize.h);
 var marker = new OpenLayers.Marker(
 new OpenLayers.LonLat(<<lon>>,<<lat>>),
 new OpenLayers.Icon(<<iconURL>>,iconSize,iconOffset)
 );
 markersLayer.setVisibility(true);
 markersLayer.addMarker(marker);
 map.addLayer(markersLayer);

Add layers to the map

Sahana Eden makes use of Openlayers and GeoExt for its GIS module. A map is basically a combination of several stitched tiles which are rendered on the web page upon request. The overall map functionalities include displaying various hospital types, locations, heliports and other map related data.We are focusing on adding a new overlay layer to the existing map layers and as well using a customised image icon for displaying marker symbol for our overlay layer.

Making the markers vary by type

As you can see, the requested functionality addition looks something like the image below, wherein the maps show different types of hospitals (clinic, emergency room, dispensary, etc) with different markers-

Design Principles

For our project, the best design principle applicable is "Open Close Principle". Since we are introducing a new module, there will definitely be extensions to it, but very few modifications.

The common features/functions will be encapsulated to ensure that the code is "DRY".

Further, we have followed the MVC(Model-View-Controller) architectural pattern to separate applications concerns.

Model - Model represents an object carrying data. It can also have logic to update controller if its data changes.

View - View represents the visualization of the data that model contains.

Controller - Controller acts on both Model and view. It controls the data flow into model object and updates the view whenever data changes. It keeps View and Model separate.

Proposed Test Cases

For end-user performance gains, Sahana Eden minimizes and compresses the CSS and JavaScript. While this approach works well for optimized end-user performance, to debug the CSS and JavaScript we shall enable debug mode in models/000_config.py in our Sahana Eden server with the following setting:

 settings.base.debug = True

Once any changes to the CSS and JavaScript are working, then we can minimizes and compresses the CSS and JavaScript using:

 static/scripts/tools/build.sahana.py

Although this uses a web service, you get better results by downloading a local version of the Closure Compiler (a tool for making JavaScript download and run faster) to static/scripts/tools. It is also possible to quickly view a single page in debug mode by adding the ?debug=1 variable to the end of a URL.