CSC/ECE 517 Fall 2011/ch4 4g ms

What is Metaprogramming?

Metaprogramming<ref>Metaprogramming</ref> can change the way you program in your programming language. This is the most under used programming techniques. Metaprogramming is writing computer programs that manipulate other programs, or that which do a part of their work at compile time that would otherwise be done at runtime. This is very advantageous because it gives greater flexibility to programs to efficiently handle new situations without recompilation. Languages which provide metaprogramming are called metalanguages. Metaprograms treat themselves as part of data and they can modify their own structure and behaviour.

History of Metaprogramming:

Not all metaprogramming involves generative programming. If programs are modifiable at runtime or if an incremental compilation is available, then techniques can be used to perform metaprogramming without actually generating source code.

Lisp first used metaprogramming by using macros as a standard facility to write code that will directly be read and compiled by the system at run time.

In C-language, one of the popular examples of meta-programming is the Lex/Yacc system for generation of parsers and lexical scanners. That is, it reads an input stream specifying the lexical analyzer and outputs source code implementing the lexer in C. Instead of doing the laborious work of generating a parser in C, one can use Yacc to generate the repetitive parts of the program. Then, one can add only the code necessary to operate on the parsed entities.

Template metaprogramming done in C++ is a metaprogramming technique in which templates are used by a compiler to generate temporary source code, which is merged by the compiler with the rest of the source code and then compiled. The output of these templates include compile-time constants, data structures, and complete functions.

Metaprogramming also exists in Java, C# and Prolog. Metaprogramming in Java comprises of three main components reflection, generics, annotations.

Uses of Metaprogramming:<ref>http://www.vanderburg.org/Speaking/Stuff/oscon05.pdf</ref>

• Metaprogramming<ref>Uses</ref> allows us to write programs that will pre-generate tables of data for use at runtime. For example, if a quick lookup table for the sine of all 8-bit integers is needed, then the sin can either be calculated and hand coded or the program can be made to build the table at startup at runtime or a program can used to build the custom code for the table before compile-time. While it may make sense to build the table at runtime for such a small set of numbers, other such tasks may cause program startup to be prohibitively slow. In such cases, writing a program to build static data tables is usually the best answer.
• A mini language can be created when there are a lot of functions which include a lot of boilerplate code. This mini language will do the boilerplate code and lets the user code only the important parts.

It is best to abstract out the boilerplate portions into a function. But often the boilerplate code isn't so pretty. Maybe there's a list of variables to be declared in every instance, or maybe there are several pieces of the boilerplate that have code inserted in certain circumstances. All of these make a simple function call impossible. In such cases, it is often a good idea to create a mini-language that allows the developer to work with the boilerplate code in an easier fashion. This mini-language will then be converted into your regular source code language before compiling. Finally, a lot of programming languages make the developer write really verbose statements to do really simple things. Code-generating programs allow the developer to abbreviate such statements and save a lot of typing, which also prevents a lot of mistakes because there is less chance of mistyping.

Examples of metaprograms are:

• Compiler or interpreter of any language
• Lex and Yacc
• CORBA's IDL compiler
• Quine
• Programs to generate EJB code and XML from database metadata
• Enhanced Cweb.

Metaprogramming in Ruby

Aspects of Ruby make it easy for metaprogramming:

Everything in Ruby is an object

This means even native types in other languages such as Integer, Float, String and even the null value (‘nil’ in Ruby) is an object. So ruby doesn’t really have functions, if every single data is an object then every single action on the data is a method.

Following example demonstrates that:

puts "cat".class
#Here's an iterator method being called on a simple Integer (Fixnum in Ruby)
5.times { puts 'hi'}
puts 5.class

#Even nil is an object!
puts nil.class

#You can even define methods in nil! We'll be looking more at this later.
def nil.hi
 puts 'hi!'
end
nil.hi

Classes and objects are open

In Ruby classes are open,it means they are open for enhancing their functionality anytime i.e even at runtime.<ref>http://www.vitarara.org/cms/ruby_metaprogamming_declaratively_adding_methods_to_a_class</ref>

This can be illustrated by following example:

Suppose we have an array [1,2,3,4,5] and we want to find the sum of elements, so we invoke method called ‘sum’ on it.

[1,2,3,4,5].sum
#It gives us ‘No method error’

So now we try to enhance the functionality of Array class by defining a method called sum on it

Class Array
 def sum
    inject(0){|s,v| s+v}
 end
end

Now lets try to invoke sum on the array

[1,2,3,4,5].sum
Output: 15

Code can be manipulated as data

Ruby code can evaluate the contents of the string at runtime, which can be used to define classes and methods on the go at runtime.<ref>http://www.slideshare.net/bsbodden/ruby-metaprogramming-08</ref>

This will be illustrated by following example:

# say_hello.rb
puts “puts ‘Hello World !!’ “

Code that invokes code

ruby –e “ ‘ruby say_hello.rb’ “
Output: Hello World !!

Ruby provides the ‘eval’ family of methods for runtime execution of code that is stored in strings

• ‘eval’ can evaluate any string. Other important methods of this family are instance_eval and class_eval.

eval(“2*2”)
Output: 4

• Instance_eval evaluates string or a code block in the context of the receiver (object).

Class MyClass
end
MyClass.instance_eval(“def hi; ‘hi’; end; “)
MyClass.hi
# ‘hi’
Obj =MyClass.new
Obj.hi
#NoMethoError : e=undefined method ‘hi’

• Class_eval evaluates an string or a code block in the context of the class or module it is invoked on.

Class MyClass
end
MyClass.class_eval(“def hi; ‘hi’; end; “)
MyClass.hi
#NoMethoError : e=undefined method ‘hi’
Obj =MyClass.new
Obj.hi
# ’hi’

Deep reflection capabilities

In Ruby it's possible to read information about a class or object at runtime. We could use some of the methods like class(), instance_methods(), instance_variables()<ref>http://www.khelll.com/blog/ruby/ruby-reflection/</ref> to do that.

Example:

# The code in the following class definition is executed immediately
 
 Class Rubyist
  # the code in the following method definition is executed later,
  # when you eventually call the method
  
  def what_does_he_do
   @person = 'A Rubyist'
   'Ruby programming'
  end
 end

 an_object = Rubyist.new
 puts an_object.class # => Rubyist
 puts an_object.class.instance_methods(false) # => what_does_he_do
 an_object.what_does_he_do
 puts an_object.instance_variables # => @person

Meta programming techniques:

Using Singleton Classes:<ref>http://ola-bini.blogspot.com/2006/09/ruby-singleton-class.html</ref>

Singleton classes are used to implement instance specific behavior in Ruby.<ref>http://www.lucasallan.com/2011/09/05/singleton-classes-on-ruby/</ref>

class MyClass 
 def blah
   'blah'
 end
end

obj = MyClass.new
obj.hi # => NoMethodError

def obj.hi
 'hi'
end

obj.hi # => 'hi'

another_obj = MyClass.new
another_obj.hi # => NoMethodError 

obj.class # => MyClass
another_obj.class # => MyClass

The above code shows one way to add instance-specific behavior.

Interesting thing to note is that the new method is invokable only from the instance it was defined on and not from other instances of the same class. This means that the method was not added to the class of the instance.

Does it mean an object itself can hold method definitions?

Well, the answer is no (unless it's an object that represents a class or a module). One of the reasons the concept of classes exists in Ruby is that they are meant to hold method definitions and act as blueprints for their instances, ie, they determine what methods can be invoked on their instances.

So where did the method 'hi' go?

It goes into the singleton class of the instance. This hidden class is different from the class the instance was created from and is a Ruby implementation detail that is typically not revealed to anyone (there is only one way to get to the singleton class, which is discussed later in the article ). This is the reason it is sometimes called ghost class or anonymous class or shadow class. The singleton class is created for an object as soon as the Ruby interpreter needs to add instance-specific behavior for that object (but not before that). Going forward, if more methods need to be added to the same object, its already-created singleton class is used. Also, more instances cannot be created using a particular singleton class (its new method raises an error). Because of this one-to-one relationship between an object and its singleton class, the word 'singleton' is used.

So, what is the way in which one can get access to an object's singleton class?

# ... in continuation from previous snippet 
obj_singleton_class = class << obj; self; end
obj_singleton_class.ancestors # => [MyClass, Object, Kernel]
MyClass.ancestors # => [MyClass, Object, Kernel]

Here class << obj; self; end is the key: anything written after class << obj and before the corresponding end is evaluated against the context of the singleton class of obj, ie, selfpoints to the singleton class.

In the above snippet, we return self, getting a reference to it in the variable obj_singleton_class. We then see that the singleton class fakes the same ancestry as the original class of the object. This implies that when a method is invoked on an object, Ruby first looks for the method definition in the object's singleton class (if it exists), if not, Ruby then continues looking up the normal inheritance hierarchy.

# ... in continuation from previous snippet
obj.hi # method looked for in singleton class; found; invoked
obj.blah # method looked for in singleton class; not found; looked for in MyClass; found; invoked
obj.xyz # method looked for in singleton class, MyClass, Object, Kernel; not found; obj.method_missing() called, raising NoMethodError

Let's quickly look at two other ways in which we could have added instance-specific behavior to an object:

class << obj
 def hi
   'hi'
 end
end

obj.instance_eval do
 
 def hi
   'hi'
 end
end

The former one should be clear from what has already been explained. The latter is tied to how def works with instance_eval (which is explained earlier in this article). Now, before concluding, let's take a step back and see if we've seen the syntax discussed so far, before. You'll probably recognize their use in defining class-level methods:

Class MyClass
 def MyClass.my_class_method1
   'class method 1'
 end
 def self.my_class_method2
   'class method 2'
 end
 class << self
   def my_class_method3
     'class method 3'
   end
 end
end

Here all class methods go into the singleton class of the object representing the class, in this case, the object referenced by the constant name:MyClass (remember classes are also objects). After all, these class method definitions need to go somewhere, and in such a way that the methods don't become part of the class's instances (ie, they aren't instance methods) nor do the methods go to the super class.

Defining methods on the fly:<ref>http://gregmoreno.ca/how-to-create-a-class-on-the-fly-in-ruby/</ref>

Ruby has the ability to define methods on the fly using ‘define_method’<ref>http://metaclass.org/2007/9/18/define-method</ref>, method_missing, ‘eval’ family methods.

Using 'eval' family methods :<ref>[1]</ref>

This is illustrated by following example.

Class Person
 def  self.make_greeting (language, greeting)
     class_eval “def greet_in_#{language}; puts ‘#{greeting}’; end”
 end

 make_greeting(“Spanish”,”Hola!”)
 make_greeting(“English”,”Hello!”)
end
alan=Person.new
alan.greet_in_English
#Hello!

p=Person
p.make_greeting(“dutch”,”Hallo!”)
lisa=p.new
lisa.greet_in_dutch
#Hallo!

The above example is used to create greetings in different languages on the go. A new greeting is created in make_greeting method, which uses class_eval to evaluate the string that defines a method for a ‘language’ (argument for the make_greeting method). As the method defined using class_val it would in the scope of the entire class (Person). Now to access greet_in_English method we need to create an object and use it to invoke the method. Now to define method at runtime we can invoke make_greeting method using the class (Person.make_greeting(language,greeting)). And now the newly defined method is available for access by any Person’s object.

The same functionality can be achieved by define method in following was:

Class Person
 def  self.make_greeting (language, greeting)
     define_method  “greet_in_#{language};” do 
       puts “#{greeting}”
     end
 end

 make_greeting(“Spanish”,”Hola!”)
 make_greeting(“English”,”Hello!”)
end

Defining methods using method_missing:<ref>http://www.trottercashion.com/2011/02/08/rubys-define_method-method_missing-and-instance_eval.html</ref><ref>http://technicalpickles.com/posts/using-method_missing-and-respond_to-to-create-dynamic-methods/</ref>

One of the ways Ruby is dynamic is that we can choose how to handle methods that are called, but don’t actually exist. If we have a lot of very similar methods, we can even use this to define them all at once. Ruby does this using the method_missing method, which we override in the classes where we need more dynamic method calling.

This can be illustrated by using following example<ref>http://kconrails.com/2010/12/21/dynamic-methods-in-ruby-with-method_missing/</ref> which creates a widget that store any attribute that is given to it.

# lib/widget.rb
Class Widget
 def method_missing sym, *args
   if sym =~ /^(\w+)=$/
     instance_variable_set "@#{$1}", args[0]
   else
     instance_variable_get "@#{sym}"
   end
 end
end

widget = Widget.new
#<Widget:0x0000010383f618>
widget.name = 'Bob'
# "Bob"
widget.age = 30
# 30
widget.name
# "Bob"
widget.age
# 30

We have a created Widget and no methods are defined in it, so whenever a method is invoked it would call method missing. In method it is looking for a command with ‘=’ symbol that means it assigning something, so if that is the case it would put that variable into the instance variable set for later retrieval. If the command doesn’t have ‘=’ it is simply displaying the attributes from the instance variable set.

Aliasing:<ref>Aliasing</ref>

To alias a method or variable name in Ruby is to create a second name for the method or variable. Aliasing can be used either to provide more expressive options to the programmer using the class, or to help override methods and change the behavior of the class or object. We can redefine a method in an existing class by opening it up and rewriting it, but since we cannot loose the behaviour provided by the original method, we can use alias_method.

alias        :new  :old
alias_method :new, :old

Syntax<ref>http://blog.jayfields.com/2006/12/ruby-alias-method-alternative.html</ref><ref>http://www.ruby-forum.com/topic/135598</ref>:

alias method-name method-name
alias global-variable-name global-variable-name

Creates a new reference to whatever old referred to. old can be any existing method, operator, global. It may not be a local, instance, constant, or class variable. Alias_method is used when there is a need to override a method but still count on its old behaviour.

Example1:

alias foo bar  -> Creates a foo alias for bar

alias $MATCH $& -> $MATCH is an alias for $&

Example2:

1.	def oldmtd  
2.	  "old method"  
3.	end  
4.	alias newmtd oldmtd  
5.	def oldmtd  
6.	  "old improved method"  
7.	end  
8.	puts oldmtd  
9.	puts newmtd  

The output is:

1.	old improved method  
2.	old method  

Local variables, instance variables, class variables and constants may not be aliased. The parameters to alias may be names or symbols. When a method is aliased, the new name refers to a copy of the original method’s body. If the method is subsequently redefined, the aliased name will still invoke the original implementation. Aliases cannot be defined within the method body. The alias of the method keeps the current definition of the method even when the methods are overridden. Making aliases for the numbered global variables ($1,$2,….) is prohibited. Overriding the built in global variables may cause serious problems.

Modules:

Module<ref>Modules</ref> is just a bunch of instance methods. A class is just a module but has certain additional features like a superclass and a new() method. The main reason for having both modules and class in Ruby is to make the code more explicit. A module<ref>http://blog.jayfields.com/2008/07/ruby-underuse-of-modules.html</ref> is used in situations where it has to be included somewhere or when it needs to be used as a namespace. A class is used when it needs to be instantiated or inherited. A module can be included by a class. A module can be shared between and included by multiple classes at the same time. Modules can be used to organize constants in the same way as directories are used to organize files.

Example<ref>http://mousebender.wordpress.com/2007/11/18/including-a-module-in-ruby/</ref>

module Stats
 def max_health
   100
 end
 def constitution
   50
 end
end

class Character
 include Stats
 attr_accessor:name
 attr_accessor:location
 def talk; end
 def walk; end
end

In the above example we have extracted max_health and constitution into a module. The Character class incudes the Stats module. Whenever a method is called on an instance of Character, that cannot be found in the referenced class, Ruby will look if the method can be found in one of the included modules. This is not the same as inheritance. Multiple modules can be included at the same time.

Modules define a namespace, a sandbox in which your methods and constants can play without having to worry about being stepped on by other methods and constants.

include mixes a module into a class or another module. Methods from that the module are called function-style (without a receiver).

Extend keyword can be used to extend a single functionality for the object. Adds to object the instance methods from each module given as a parameter.

module Mod
 def hello
   "Hello from Mod."
 end
end

class Klass
 def hello
   "Hello from Klass."
 end
end

k = Klass.new
k.hello	»	"Hello from Klass."
k.extend(Mod)	» #<Klass:0x401b598c>
k.hello	» "Hello from Mod."

extend is used to include a module in an object(instance). Methods in the module become methods in the object

The possibilities meta programing opens up seem to be endless. Joining on the way are endless possibilities to do something wrong. “with great power comes great responsibility”

References

<references/>

Further Reading

More About Metaprogramming

More about Singleton Classes