CSC/ECE 517 Fall 2012/ch1 1w10 pk

Introduction

In the previous article for which the link does not exist yet, we learn that metaprogramming is where the computer code interacts with other programs as data and performs many of these interactions at compile time rather than runtime. This built-in interaction allows the programmers to take advantage of these capabilities and focus their time on the rest of their program logic instead of the details of some of the lower level coding. In this article, we will take a closer look at one of the styles of metaprogramming referred to as language extensions for object-relational mapping through examples and some comparisons of language extensions and tools.

Object-Relational Mapping

Object-Relational Mapping (ORM) is a methodology for managing data between object oriented systems and relational databases. The premise of the concept is to provide a universal method to access data in a database. This is beneficial for all programming languages that can use objects to store and retrieve data from the database. There are many languages that are using ORM as a technique to manage database data. In the rest of the article, we will discuss ORM and some of the languages in use today.

Language Extensions

Language extensions for ORM have often been traditionally classified as design patterns or software tools used to perform basic create, read, update and delete (C.R.U.D.) operations on relational databases, that is, until recent new approaches such as ActiveRecord have grown in popularity. ActiveRecord is not just a design pattern it is an increase of function to the active record pattern approach by adding inheritance and associations. Examining ActiveRecord and other language extensions will allow for comparisons of the ease of programming using these language extensions verses the conventional database oriented systems approach.

Ruby and ActiveRecord

All too often programmers are faced with the challenge of persisting objects from their program into a datastore. Custom code is created for this purpose that can be complex or difficult for others to understand as well as not seem natural. Applications that are designed to persist data have the need to know how the objects correspond to the information stored in these database tables. Ruby on Rails, first released in 2005, is able to provide a uniform method to help resolve these complicated issues without sacrificing function or knowledge of these objects by using ActiveRecord (AR). AR is a persistence engine that comes as part of Ruby on Rails. "It creates a 'persistable' domain model from business objects and database tables, where logic and data are presented as a unified package" 3. AR is admired for its simplistic and elegant approach of removing these levels of complexity. It allows for a 'pluggable' solution to many different popular databases available today to include: MySQL, SQLite, SQL Server, PostgreSQL, and Oracle.

ActiveRecord uses a Single Table Inheritance to allow for inheritance capabilities and provides a set of macros for association relationships between classes of objects, such as belongs_to, has_one, has_many, etc. AR is not only a component of the Model view-controller (MVC) for Ruby on Rails but it is also a standalone ORM package for Ruby itself. Ruby, Ruby on Rails, and ActiveRecord continue to grow in popularity due to not only the curiosity of programmers but their ability to improve function and feature sets while maintaining the initial intent of the language, "trying to make Ruby natural, not simple" -- Yukihiro “matz” Matsumoto 8. Other languages have been able to learn from AR and have tried to add this capability for themselves. Let’s take a closer look at some other implementations that attempts to duplicate AR’s elegance.

Other Examples of Language Extensions

ADO.NET Entity Framework

ADO.NET Entity Framework, Microsoft's ORM, part of .NET 4.0 was first developed in 2008. The Entity Framework tries to remove ORM mismatches that often plague conventional database oriented programs by using an Entity data model (EDM) and an Entity-Relationship data model to define the Relationships associated to each Entity. The EDM consists of a schema and a mapping specification. The schema defines the data types from the entities and the mapping provides the connections between the database scheme and the conceptual scheme. The Entity Framework has its own version of a query language called Entity SQL which continues to focus only on the conceptual entities and relationships as opposed to the actual database. It works with many popular databases available today to include: MySQL, SQLite, SQL Server, PostgreSQL, and Oracle.

Django

Python, presents an interesting case because it is an extremely similar language to Ruby in syntax and somewhat in philosophy. Django, is an ORM included in Django open source framework for Python. It follows a MVC architectural pattern and was originally released in 2005 with the primary goal of making complex, database driven websites easier to create and maintain. Instilled in its principles are emphasis on loose coupling, rapid development, don’t repeat yourself, and reusability (a.k.a. less code). The basis of its design is to encapsulate objects just like the ActiveRecord design pattern in models so that all the information needed can be stored in the model and knowledge of the database is not exposed. Django can essentially be considered the equivalent to Rails for Ruby.

Enterprise Objects Framework

Lastly we will introduce Enterprise Objects Framework (EOF), Mac OS X/Java, part of Apple WebObjects. EOF is the oldest of the bunch, introduced in 1994 for a product call NeXTSTEP. It was indented to eliminate the interaction with the relational database and the Java or Objective-C objects. EOF was later integrated into WebObjects. An EOFModel contains the mappings from the database to the classes, class attributes, and objects. EOF also incorporates inheritance into its feature set to allow a more object oriented approach using Enterprise Objects to reflect the hierarchy. EOF uses Java Database Connectivity (JDBC) and Java Naming and Directory Interface (JNDI) to talk to databases that support that, such as Oracle, DB2, and MySQL.

Java's Hibernate

Hibernate is Java’s ORM framework library that maps Java classes to database tables and provides data query and retrieval functionality via XML mappings done though Java Annotations configurations. In addition, it can generate SQL calls that lessen the need for developers to convert result sets into objects manually. One of the first things that Hibernate uses for any interactions is a Session Factory class which is designed as a class factory to return the current Hibernate session object and ensure that only one instance of the session be issued per thread. Without listing the actual code of the Session Factory the code basically allows for getting the session object, opening the session for use, and closing the session after use.

Unfortunately, Hibernate database table creation can only be done manually using separate tools or additional Java code. This is a prime example of the additional knowledge a developer must have to correctly create a hibernate project. A developer must know that each object type must have its own table and that the fields within the table must correspond to the types of the object’s member variables.

Based on the tables created, classes must then be created with the member variables and setters/getters associated with the field types created in the database. For example, if the database table contains a field 'name' which is of type varchar and limited to 255 characters, the setter/getter must be of type String and the coder (if they wish) may limit the entry in the setter to 255 characters.

The next step is to create a Hibernate configuration file which essentially defines the database connection including the url, username, database type, password, etc. After this, the developer must explicitly tell Hibernate the relationships between the member variables of the class and the fields of the database with either an xml file or using Java annotations inside of the class. This is yet another example of the extra effort that has to be taken by the developer, but which also gives the developer great power and flexibility to name the fields and member variables different things.

JazzRecord for JavaScript

JavaScript presents another good case study for the use of ORM and ActiveRecord in application development. For one, JavaScript, like Python and Ruby, is loosely typed which can make dealing with the strictness of database typing an interesting challenge for these languages. Secondly, JazzRecord was modeled specifically after ActiveRecord in Ruby but does have its own differences.

Programming using Language Extensions

Several language extensions have been introduced in previous sections of the article. Let's take a closer look at a specific scenario and show how each of them would implement their respective table for multiple language packages. In some of the examples below you will see that many of the language extensions require external tools to generate the hooks in to the language.

To illustrate the differences in implementation of ORM in different language packages, we will use the following example: Products and Categories.

A Category can have many Products but a Product can only have one Category. A sample query would be

SELECT product.productName & product.unitPrice FROM Products WHERE product.category.categoryName = “Beverages” 

ActiveRecord

For ActiveRecord in Ruby on Rails, the syntax to create the table is contained within the Product class that inherits from the ActiveRecord class. No external tools are required to generate the Ruby code, only the Ruby interpreter.

class Product < ActiveRecord::Migration
  def self.up
    create_table :product do |p|
      p.string :name
      p.string :id
      p.string :category
      p.float :unitPrice
    end
  end
  
  def self.down
    drop_table :product
  end
end

Now create a new Product in Ruby, update the attributes, and save the object. The .save method will actually update the database table.

n = Product.new
n.id = "1"
n.name = "Coke"
n.category = "beverage"
n.unitPrice = 0.99
n.save

ADO.NET Entity Framework

ADO.NET requires that you use the Entity Model Designer which is part of Microsoft Visual Studio. The Designer allows you to create your model which includes any entities and associations.

The Designer will ultimately generate an XML file that represents the Entity Data Model.

<!-- A snippit from the xml file – shows the Product definion -->
<!-- There are not associations in out example -->
<EntityType Name = "Product">
     <Property Name = "Name" Type = "String" /> 
     <Property Name = "Category" Type = "String" /> 
     <Property Name = "Unit Price" Type = "Float" />           
     <Property Name = "id" Type = "String" Nullable = "false" />
</EntityType>
 

In code for the Product object, the setter on the class would look something like this.

set
{
   this.CategoryReference.EntityKey = new EntityKey("ProductEntities.Category", "id", "name", "unitPrice", "category");
}

Django

In Django, a model is defined to represent a product.

  from django.db import models
  class Product(models.Model):
    name= models.CharField(max_length=200)

After creating a model for a product it will generate a corresponding table. Note that "id" is included as part of the model, so this might cause some unexpected issues if you planned to use id differently.

  BEGIN;
  CREATE TABLE myapp_product (
    "id" serial NOT NULL PRIMARY KEY,
          "name" varchar(200),
          "category" varchar(200),
          "unitPrice" decimal
  );

Create a new instance of a Product and save it.

p =  Product(name='Coke', category='beverage', unitPrice=0.99)
p.save()
p.id     # Returns the ID of your new object.

Enterprise Objects Framework

The biggest downside with EOF and WebObjects is that it is not free. The Server and development environment is fairly expensive. However, here is a quick look at how you would create an EOF model that represents the classes, attributes and the objects.

To generate the Product definition developers must use the EOModeler UI.

The code to create a new instance / database entry is fairly intuitive.

   EOEditingContext ep =  new EOEditingContext();

   Product p =  new Product();
   p.id = "1";
   p.name = "Coke";
   p.category = "beverage";
   p.unitPrice = 0.99;
 
   ep.add(p);
   ep.saveChanges();

Java Hibernate

There is no question that the set up effort involved in getting a Hibernate project going is substantially more than in ActiveRecord for Rails. The actual code to create and manipulate the objects is approximately the same between the two languages. This is clearly a case where more power is given to the developer but it is not without cost, which in this case is a fairly complex and involved list of set up steps.

Below is an example set up for a Hibernate transaction

Product p = new Product();
p.name("Coke");
p.category("Beverage");
p.unitPrice(0.99)

Transaction tx = null;
Session s1 = SessionFactory.getInstance().getCurrentSession();
try 
{
   tx = session.beginTransaction();
   s1.save(p);
   tx.commit();
}
catch(RuntimeException e) 
{
   tx.rollback();
}

JStORM for Javascript

Example adapted from Ajaxian

JStORM is very similar to Rails in which we define a Model that will hold our Products

var Product = new JStORM.Model({
  name:"Product", // the first argument is the table name to use in the db.
  fields: { // the second argument is an object that defines the table columns
// the keys are the column names and the values are the column types (limited to Integer,Float,TimeStamp and String)
// The additional sql properties can be defined: notNull, defaultValue, maxLength, unique
     productName:new JStORM.Field({type:"String",maxLength:25}),
     category:new JStORM.Field({type:"String", maxLength:25}),
     unitPrice:new JStORM.Field({type:"Float"}),
  },
  connection:"default"
});

Basic SQL operations are one line code statements

Product.createTable(); // creates the table
Product.insert(["Coke","beverages"]); // insert Coke beverage into Product table
Product.insert(["PBR","beverages"]); // insert PBR
Product.select("category = 'beverages'") // selects products of category beverages

Jazz Record for Javascript

Declaring a model in JazzRecord is very similar to ActiveRecord in structure, where the types of each of the fields must be explicitly identified so that the correct table can be created.

var Products = new JazzRecord.Model({
  table: "products",
  columns: {
    name: "Coke",
    category: "beverage",
    unitPrice: 0.99
  }
});

Like ActiveRecord, retrieving objects from the database is effortless and very transparent. Here is an example of querying a few objects and then using those objects.

var product1 = Products.findBy("name", "Coke");
var product2 = Products.findBy("category", "beverage");
var sameProduct = (product.name == product2.name);
product2.name = "NewProduct";
product2.save();

The JazzRecord team even decided to follow ActiveRecord in including the ability to identify associations between the different models. The following is an example of how to define associations between two models:

//new version of JazzLegend model
Category = new JazzRecord.Model({
  table: "categories",
  foreignKey: "product_id",
  hasMany: {products: "products"},
  columns: {
    name: "beverage"
  }
});

//associated model
Product = new JazzRecord.Model({
  table: "products",
  belongsTo: {category: "categories"},
  columns: {
    name: "text",
    unitPrice: 0.99,
    category_id: "1"
  }
});

ODB for C++

Example adapted from Code Synthesis

 #pragma db object
  class product
  {
    ...
  private:
    product ();

    #pragma db id auto
    unsigned long id_;

    string productName_;
    string category_;
    float unitPrice_;
  };

Given the above declarations, we can perform various database operations with objects of the product class:

 product coke ("Coca-Cola", “beverages”, 0.99);
 product pbr ("Pabst Blue Ribbon", “beverages”, 1.99);
 odb::transaction t (db.begin ());
db.persist (coke);
db.persist (pbr);

  typedef odb::result<product> result;
  typedef odb::query<product> query;
  result r (db.query<product> (query::category = “beverages”));
  copy (r.begin (), r.end (), ostream_iterator<price> (cout, "\n"));

  coke.unitPrice (coke.unitPrice () - 0.50);
  db.update (coke);

  t.commit ();

Anito for .NET

Example adapted from anito.NET

First, we instantiate a DataSession

// provider for MS-SQL
Anito.Data.SqlClient.SqlProvider provider = new Anito.Data.SqlClient.SqlProvider();
provider.ConnectionString = "ConnectionString";
ISession session = new DataSession(provider);

To insert data into our new database table

Product product = new Product();
product.ProductCode = "ENT00001";
product.Name = "Coke";
product.Category = “beverages”;
product.UnitPrice = “0.99”;
session.Insert<Product>(product);

To update a row

Product product = session.GetT<Product>(p => p.ProductCode == "ENT00001");
product.Name = "Coca-Cola";
session.Update<Product>(product, x => x.ProductCode == "ENT00001");

Assuming that Product is an anito mapped object. We can perform a Select query several ways

//Get by key
Product product1 = session.GetT<Product>("PRD-000123");

//Get by expression
Product product2 = session.GetT<Product>(x => x.ProductCode == "PRD-000123");

//List
List<Product> list1 = session.GetList<List<Product>, Product>();

List<Product> list2 = session.GetList<List<Product>, Product>(p => p.ID < 100);

//Paged
List<Product> pagedList = session.GetPagedList<List<Product>, Product>(10, 3);

To remove data from a table we run the Delete command

session.Delete<Entity>(p => p.ProductCode == "ENT00001");

LINQ to SQL for C#

The query we are from above, in C#, is as follows:

var results = database.Products.Where(product => product.Category.CategoryName == "Beverages")
                              .Select(product => new
                                                     {
                                                         product.ProductName,
                                                         product.UnitPrice
                                                     });

There is more than one mechanism for implementing ORM in C#. One of the easiest ways is to create models in C# from SQL schemas is to use Visual Studio (http://www.microsoft.com/visualstudio/eng/launch-day). Using this method requires SQL Server be installed.

In order to complete this query, mapping between the SQL databases, tables and fields and C# object models must be created. The first step is to set up a data connection to the database you are trying to use to create the model. Once the connection has been established, you need to create a “LINQ to SQL Classes” item in the project. Once the item is created, this creates a [DBName].dbml (for our example, we will call the DB 'Store') file and the O/R designer is opened. From here, you can drag and drop the tables of interest into to O/R designer. Going back to the Product/Category example, there should be two boxes (one for each table) and an arrow in between showing the relation between the two tables (in our case, one-to-many).

Once this is done, the object models have been created and the designer has automatically generated classes to represent records in the tables of interest. In the Product/Category example, the following classes would be generated with the following information:

[Table(Name = "dbo.Categories")]
public partial class Category : INotifyPropertyChanging, INotifyPropertyChanged
{
   private int _CategoryID;
   private EntitySet<Product> _Products;

   [Column(Storage = "_CategoryID", AutoSync = AutoSync.OnInsert,
       DbType = "Int NOT NULL IDENTITY", IsPrimaryKey = true, IsDbGenerated = true)]
   public int CategoryID
   {get
       { return this._CategoryID; }
       set
       { if ((this._CategoryID != value))
           {
               this.OnCategoryIDChanging(value);
               this.SendPropertyChanging();
               this._CategoryID = value;
               this.SendPropertyChanged("CategoryID");
               this.OnCategoryIDChanged();
           }
       }
   }

  Category class: represents each record in Categories table;
       CategoryID property (an int): represents the CategoryID field; So are the other properties shown above;
       Products property (a collection of Product object): represents the associated many records in Products table

[Table(Name = "dbo.Products")]
public partial class Product : INotifyPropertyChanging, INotifyPropertyChanged
{
   private int _ProductID;
   private Entity<Category> _Category;

   [Column(Storage = "_ProductID", AutoSync = AutoSync.OnInsert,
       DbType = "Int NOT NULL IDENTITY", IsPrimaryKey = true, IsDbGenerated = true)]
   public int ProductID
   {get
       { return this._ProductID; }
       set
       { if ((this._ProductID != value))
           {
               this.OnProductIDChanging(value);
               this.SendPropertyChanging();
               this._ProdcutID = value;
               this.SendPropertyChanged("ProductID");
               this.OnProductIDChanged();
           }
       }
   }

   Product class: represents each record in Products table;
        ProductID property (an int): represents the ProductID field; So are the other properties shown above;
        Category property (a Category object): represents the associated one records in Products table, and;

[Database(Name = "Store")]
public partial class StoreDataContext : DataContext
{ public Table<Category> Categories
   {get
       { return this.GetTable<Category>(); }
   }

   public Table<Product> Products
   { get
       { return this.GetTable<Product>(); }
   }
}

   StoreDataContext class: represents the database;
       Categories property (a collection of the Category objects): represents the Categories table;
       Products property (a collection of the Product objects): represents the Products table;

The next step is to map the SQL representations to the C# representations. If you view the Store.dbml file generated by Visual Studio, you will see that is is an XML file. This XML file contains the SQL to C# mappings. There is also a Store.dbml.layout file which outlines how the designer should visualize the object models. Finally, a Store.designer.cs file is created that contains the C# “partials” that were generated.

If needed, the mappings can be easily customized in the designer as well. Any modifications made in the designer are automatically made in the XML files / C# code. The final mapping is a one way mapping: from SQL Server to C#.

It is also possible to do the same thing on the command line using SQLMetal. With SQLMetal, you can do a number of things, including creating the .dbml file created by the Visual Studio designer. The command to run SQLMetal is sqlmetal [options] [<input file>].

Java Persistence API

The Java Persistence API can be used for object-relational mapping in Java. It is POJO-based, meaning it is based solely on plain-old Java objects (POJO). This API uses “entities” that are written as POJOs and marked using the @Entity annotation. An entity is a persistence domain object that represents a table in a relational database. An entity of our Category table would look something like this:

@Entity
  @Table (name="Category")
  public class Category implements Serializable {
     @Id
     @Column(name="id", nullable=false)
     private String CategoryId;
     @Column(name="name")
     private String CategoryName;

The class would also include getters and setters for the various fields. The @Table annotation denotes the database table represented by the entity however if the table and entity name is the same, there is no need for the @Table annotation. The @Column annotation denotes the database column that corresponds to the property/field. The naming assumption applies here as well.

The @ID annotation is used to mark a field/property as a primary key. For our ongoing example, it would something look like this:

     @Id
     public String getCategoryId() {
        return this.deptId;
     }

     public void setCategoryId(String deptId) {
        this.deptId = deptId;
     }

Entity state can be represented using embedded classes; these do not have persistent objects of their own.

Depicting relationships has become easier with the Java Persistence API; annotations are used that can specify multiplicity as well as relationship direction. In our ongoing examples, we can implement a many-to-one relationship as follows:

@Entity
  public class Product implements Serializable {

     private Category category;
     @ManyToOne
     public Category getCategory() {
        return Category;
     }

     public void setCategory (Category category){
        this.category = category;
     }
     .........
  }
 

  @Entity
  @Table (name="Category")
  public class Category implements Serializable {
     .........
     private Collection<Product> products = new HashSet();
     .........
     @OneToMany(mappedBy="category", fetch=FetchType.EAGER)
     public Collection<Product> getProducts() {
        return products;
     }

     public void setProducts(Collection<Product> products) {
        this.products = products;
     }
     .........
  }

If there were a bidirectional relationship between Product and Category, the element mappedBy could be used when specifying multiplicity in the relationship's inverse side by pointing to the name of the property/field that owns the relationship.

Conclusion

Depending on what language you plan to use for your application will dictate what extensions and tools you will use. If you are driven by cost, then some of the choices are obvious to remove, otherwise you might be driven by you familiarity with a particular language or existing tool set. Either way one can see how all of the languages have attempted to simplify the process of accessing data in an external datastore. There is no clear victor in these comparisons, and we should believe the situation will dictate which language to use.

What’s Next?

In the next article for which the link does not exist yet, we will look at another style of metaprogramming, CRC Cards, one of the simplest methods of doing object oriented design. These cards are a way to capture the initial relationships between objects for a given system. CRC cards will help determine the evolution of these relationships before any code is written. You can learn very quickly about CRC cards by taking a close look at the software used to create CRC cards, and other artifacts of o-o design.

References

1. Comparison of web application frameworks
2. List of object-relational mapping software (Wikipedia.org)
3. ActiveRecord Ruby (Wikipedia.org)
4. ADO.NET Entity Framework (Wikipedia.org)
5. Django (Wikipedia.org)
6. Enterprise Objects Framework (Wikipedia.org)
7. Programming Ruby - The Pragmatic Programmer's Guide
8. Ruby – A Programmers Best Friend
9. Django Project (Djangoproject.com)
10. ADO.NET (Micorsoft.com)
11. Enterprise Objects Framework (Apple.com)