CSC/ECE 517 Spring 2014/ch1 1w1d mm

Serialization

Serialization[1] is a process of converting a data structure or an object into a stream of bytes or string to facilitate storage in memory, file(persistence storage) or transmission over a network. The process of Serialization is also referred to as Marshalling[2]. The stream of data has to be in a format that can be understood by both ends of a communication channel so that the object can be marshaled and reconstructed easily.

Basic Advantages of Serialization:

1. Communication between two or more processes on same machine. Object state can be saved and shared in a persistent or in-memory store.

2. Communication between processes on different machines. Serialization facilitates the transmission of an object through a network.

3. Creating a clone of an object.

4. Cross-platform compatibility. Object can be serialized in a common format that is understood by multiple platforms. Eg. JSON, XML.

De-serialization is the process of converting the stream of bytes or string back to objects in memory. It is the process of reconstructing the object later.This process of de-serialization is also referred to as Unmarshalling.

Few Practical Applications for Serialization

1. HTTP Session Replication by sharing session objects across web servers for handling failover scenarios

2. Serialization facilitates communication in Remote Method Invocation or Remote procedure calls

3. Rails Cookie Handling[3]. Cookies are stored marshalled/unmarshalled to and from client machines.

Serialization in Ruby:

Ruby provides serialization capabilities through its module, Marshal. There are also other libraries like YAML and JSON which can be used in Ruby to generate serialized objects for purposes like platform independence and human readable formats.

Types of Serialization

Serialization in Ruby can be done in two ways. During serialization, the object in memory can be converted into Human Readable formats like YAML (YAML Ain’t Markup Language) and JSON (JavaScript Object Notation), or the object can be converted into binary format.

Converting Ruby Objects in Human Readable Formats

The conversion of Ruby objects into YAML and JSON formats are explained below.

Converting Ruby Objects to YAML format

YAML[4] format is a human friendly data serialization standard for all programming languages. YAML (YAML Ain't Markup Language) is perhaps the most common form of serialization in Ruby applications. It is used for configuration files in Rails and other projects, and is nearly ubiquitous. YAML is a plaintext format, as opposed to Marshal's[5] binary format. Objects stored as YAML are completely transparent and editable with nothing more than a text editor. It also has a simple, spartan syntax that's easy to look at and easy to type. It is not encumbered by excessive wordage and symbols seen in XML. In order to use it in Ruby, the yaml.rb file is required to be loaded which provides methods for converting objects into yaml format and creating .yml files.

Examples of serialization using YAML:

    #Serialization using YAML's to_yaml method
 
  require "yaml"
     class First
     def initialize(name, age, country)
	@name = name
	@age = age
	@country=country
     end
      def to_s
	"In First:\n#{@name}, #{@age}, #{@country}\n"
     end
   end
 
  x = First.new("Tom", 25, "USA")
  puts x
  puts x.to_yaml

Output:

  In First:
  Tom, 25, USA
  --- !ruby/object:First
  name: Tom
  age: 25
  country: USA

The above code displays the object x, first as a string and then in the yaml format.

Saving YAML data into a file:

 # Serialization using YAML::dump

require 'yaml'

f = File.open( 'first.yml', 'w' )
YAML.dump( ["Tom", 25, "USA"], f )
f.close
		
File.open( 'first.yml' ){ |f|	
    $arr= YAML.load(f)
}	

p( $arr )

Output:

  ["Tom", 25, "USA"]

The dump function can be used to serialize the data and save it into a file in YAML format. As shown in the above example the data in YAML format can be de-serialized using the load function.

YAML libraries also provides an option of selecting only those variables of the object that are needed to be serialized. This is done using the to_yaml_properties method as shown in the below example.

   #Custom Serialization using YAML
 
  require "yaml"
     class First
     def initialize(name, age, country)
	@name = name
	@age = age
	@country=country
     end
     def to_s
	"In First:\n#{@name}, #{@age}, #{@country}\n"
     end
      def to_yaml_properties
	   ["@name","@age"]  #@country will not be serialized
     end
   end
 
  x = First.new("Tom", 25, "USA")
  puts x
  puts x.to_yaml

Output:

  In First:
  Tom, 25, USA
  --- !ruby/object:First
  name: Tom
  age: 25

Converting Ruby Objects to JSON format:

JSON[6] is a light-weight data interchange format. JSON is typically generated by web applications and can be quite daunting, with deep hierarchies that are difficult to navigate. Any Ruby object can easily be serialized into JSON format. Ruby 1.8.7 distribution is not bundled with json gem. However, in Ruby 1.9.2, the json gem is bundled with the core Ruby distribution. The JSON library can be installed using Ruby Gems[7] as shown below:

# gem install json

A JSON string for serialization can be created by using the JSON.generate method:

       require 'json'
       my_hash = {:Welcome => "Ruby"}
       puts JSON.generate(my_hash) => "{\"WELCOME\":\"RUBY\"}"

Output:

{"{\"Welcome\":\"Ruby\"}"=>"{\"WELCOME\":\"RUBY\"}"}

A JSON string received from another program can be parsed by using JSON.parse Ruby thus converts String to Hash.

       require 'json'
       my_hash = JSON.parse('{"Welcome": "Ruby"}')
       puts my_hash["Welcome"] => "Ruby"

Converting Ruby Objects to Binary Formats

Binary Serialization is another form of serialization in Ruby which is not in human readable form. Binary Serialization is used when high performance serialization and de-serialization process is required and when the contents are not required to be in readable format. Binary Serialization is done using Marshal which is built into Ruby and the code for it is written in Ruby's Marshal module(marshal.c) and thus no additional files are required in order to use it. The _dump and _load methods defined in marshal are used for serialization. Using marshal module the following types of objects can not be serialized: bindings, procedure objects, singleton objects, instances of IO objects and interfaces.

Since the Binary Serialized data is not in human readable form, the following guidelines need to be followed.

    1.Use print instead of puts[8] when serialized objects are written to a file in order to avoid new line characters to be written 
      in the file.
    
    2.Use a record separator in order to differentiate between two objects.

Binary Serialization Example:

   class Animal
     def initialize  name, age
     @name = name
     @age=age
   end
  end
  class Cat < Animal
   def to_s
    "In Cat C: #{@name} \t #{@age}"
   end
  end
 c = Cat.new("Kitty Kat",5)
 puts "Before Serialization"
 puts c
 #puts d
 serialize_cat= Marshal.dump(c) #dumps the serialized cat object into serialize_cat
 puts "\nAfter Serialization:\n #{serialize_cat}"
 deserialize_cat= Marshal::load(serialize_cat) #deserializes the cat object and loads it back into deserialize_cat
 puts "\nAfter Deserialization\n #{deserialize_cat}"

Output:

  Before Serialization
  In Cat C: Kitty Kat 	 5
  After Serialization:
  oCat:
  @nameI"Kitty Kat:ET:	@agei
  After Deserialization
  In Cat C: Kitty Kat 	 5

Similar to YAML, Marshal can also be used to dump data into a file. The above example showing the serialization using YAML::dump can be written using marshal as shown below.

   f = File.open( 'first.yml', 'w' )
   Marshal.dump( ["Tom", 25, "USA"], f )
   f.close	
   File.open( 'first.yml' ){ |f|	
   $arr= Marshal.load(f)
   }	
   p( $arr )

Output:

  ["Tom", 25, "USA"]

Notice that in the above example there is no "require" statement as opposed to the earlier example of writing serialized data into files using YAML. This is because unlike YAML, marshal is built-in Ruby and no external libraries is required in order to use its functionality.

Marshal can also be used to custom serialize an object i.e. it provides an option to omit the variables of an object that are not required in the serialized data. The following program shows the use of marshal_dump method for achieving custom serialization.

     class First
     def initialize(name, age, country)
        @name = name
        @age = age
        @country=country
     end
     def to_s
           "In First: #{@name}, #{@age}, #{@country}"
     end
     def marshal_dump
           [@name,@age]  #@country will not be serialized
     end
     def marshal_load(data)
      @name=data[0]
      @age=data[1]
      @country="United States of America"
     end
     end

     x = First.new("Tom", 25, "USA")
     puts x
 
     marshal_data = Marshal.dump( x ) 
     y = Marshal.load( marshal_data ) 
     p( y.to_s )

Output:

   In First: Tom, 25, USA
   "In First: Tom, 25, United States of America"

Serialization Performance in Ruby

The different serialization formats described above differ in the efficiency at which they can serialize and deserialize data and thus while serializing large amount of data their efficiency is taken into account. The .report method in Ruby can be used to evaluate the performance of these serialization patterns. Marshal, as it handles data in binary format, is the most efficient form of serialization in Ruby. The following example compares the performance of marshal, JSON and YAML format while processing a large array and a large hash.

  require 'benchmark'
  require 'rubygems'
  require 'json'
  require 'yaml'
  include Benchmark

  benchmark_iterations = 1
  large_single_dimension_array = [42, 123.123] * 5000
  large_single_dimension_hash  = {}
  10000.times do |i|
    large_single_dimension_hash["key#{i}".intern] = i
  end

  class Fixnum
    alias to_ruby to_s
  end

  class Float
    alias to_ruby to_s
  end

  class String
    alias to_ruby inspect
  end

  class Symbol
    def to_ruby
      ":#{to_s}"
    end
  end

  class Array
    def to_ruby
        "[#{map {|x| x.to_ruby}.join(',')}]"
    end
  end

  class Hash
    def to_ruby
      "{#{map {|k, v| "#{k.to_ruby} => #{v.to_ruby}"}.join(", ")}}"
    end
  end

  benchmark do |t|
    t.report("array marshal") do
      benchmark_iterations.times {
        Marshal.load(Marshal.dump(large_single_dimension_array))
      }
    end
    t.report("array json") do
      benchmark_iterations.times {
        JSON.load(JSON.dump(large_single_dimension_array))
      }
    end
    t.report("array yaml") do
      benchmark_iterations.times {
        YAML.load(YAML.dump(large_single_dimension_array))
      }
    end
    t.report("hash marshal") do
      benchmark_iterations.times {
        Marshal.load(Marshal.dump(large_single_dimension_hash))
      }
    end
    t.report("hash json") do
      benchmark_iterations.times {
        JSON.load(JSON.dump(large_single_dimension_hash))
      }
    end
    t.report("hash yaml") do
      benchmark_iterations.times {
        YAML.load(YAML.dump(large_single_dimension_hash))
      }
    end
  end

Output:

 array marshal  0.000000   0.000000   0.000000 (  0.002228)
 array json  0.010000   0.000000   0.010000 (  0.011091)
 array yaml  0.410000   0.010000   0.420000 (  0.423433)
 hash marshal  0.030000   0.000000   0.030000 (  0.028195)
 hash json  0.020000   0.000000   0.020000 (  0.023310)
 hash yaml  0.860000   0.010000   0.870000 (  0.880581)

The output of the above example shows that, given large chunks of data, marshal is comparatively more efficient format for serialization in Ruby.

Serialization in OOLS Languages: Comparison

Sl.No	Ruby	Java	.Net Framework	C++
1	Ruby provides a built in module called Marshal for serialization	Java uses an Interface named Serializable interface for classes to implement	.Net provides a Serializable Attribute	Although, there is no built in support for serialization in C++, it can be achieved by using Boost libraries[9]
2	The built in module of Ruby (Marshal) does not support platform independence, however, it can be achieved by using external libraries like YAML and JSON	Similarly, Java's built in serialization is also not platform independent and in order to use serialization in Java across Ruby platform, jruby library should be used.	.Net used Remoting technology to make it platform independent.	Serialization using the Boost libraries is not platform independent.
3	YAML provides a method (to_yaml_properties) which can be used to select the variables who's value is need to be serialized. With Marshal, we need to write a method named marshal_dump defining the variables of an object that has to be serialized.	Provides an option for serializing only the required attributes to be serialized for an object. Use the keyword Transient to ignore certain data that doesn’t need to be serialized	XML Serializer sets XmlIgnoreProperty to true to ignore the default serialization of a field or a property	Serialization using the Boost libraries is custom and thus the user can specify the part of the objects to be serialized.
4	Bindings, procedure objects, singleton objects, instances of IO objects and interfaces can not be serialized. Serializing these objects throws TypeError exceptions.	Thread, OutputStream and its subclasses, and Socket are not serializable in Java.[10]	Objects like DataRow are non serializable in .NET. Whether an object is serializable or not can be checked using the Type.IsSerializable	There are no such objects in C++.