CSC/ECE 517 Fall 2012/ch1b 1w53 kc

Metaprogramming in dynamically typed languages

---

Introduction

One of the most under-used programming techniques is writing programs that generate programs or program parts. Code-generating programs are sometimes called metaprograms; writing such programs is called metaprogramming. Writing programs that write code has numerous applications. In this article, we will learn why metaprogramming is necessary and look at some of the components of metaprogramming in dynamically typed languages.

What is Metaprogramming?

Metaprogramming is the writing of computer programs that write or manipulate other programs (or themselves) as their data, or that do part of the work at compile time that would otherwise be done at runtime. In some cases, this allows programmers to minimize the number of lines of code to express a solution (hence reducing development time), or it gives programs greater flexibility to efficiently handle new situations without recompilation. Typically, you use a metaprogram to eliminate or reduce a tedious or error-prone programming task. So, for example, instead of writing a machine code program by hand, you would use a high-level language, such as C, and then let the C compiler do the translation to the equivalent low-level machine instructions.

Metaprogramming Example

Let us consider one big main function with 1,000 printf instructions.

   #include <stdio.h>
   int main(void) {
     printf("1. I must not chat in class.\n");
     printf("2. I must not chat in class.\n");
     /* 996 printf instructions omitted. */
     printf("999. I must not chat in class.\n");
     printf("1000. I must not chat in class.\n");
     return 0;
   }

Taking the above pseudo code example, we can see how that can be made much simpler and elegant using a metaprogram in Ruby.

   File.open('punishment.c', 'w') do |output|
     output.puts '#include <stdio.h>'
     output.puts 'int main(void) {'
   1.upto(1000) do |i|
     output.puts "    printf(\"#{i}. " +
     "I must not chat in class.\\n\");"
   end
     output.puts '    return 0;'
     output.puts '}'
   end

This code creates a file called punishment.c with the expected 1,000+ lines of C source code

Dynamically Typed Programming Languages

Before defining what dynamic typing is, it is easiest to define its opposite. Statically typed languages are those which define and enforce types at compile-time. Statically typed languages typically have clearly distinct compile-time and run-time phases, with program code converted by a compiler into a binary executable which is then run separately. Dynamically typed languages have distinct compilation and execution phases and therefore we use the terms compile-time and run-time identically for both statically and dynamically typed languages. In most dynamically typed languages (e.g. Ruby, Perl, and Python) ‘running’ a file both compiles and executes it.

Metaprogramming in Dynamically Typed Languages

Metaprogramming can not only be used for programs to write programs, it can also be used to manipulate itself at runtime. Let us look at some examples that do this in ruby and the equivalent code in Javascript and Python.

Adding a method dynamically to a class

We can reopen an already existing class/prototype and add a new method to it dynamically. In ruby we reopen the class simply using the class keyword. In Javascript we grab the prototype for our object and just attach a new function.

Ruby

 class Ninja
   def battle_cry
   puts "#{name} says zing!!!"
   end
 end
 drew.battle_cry
 # => Drew says zing!!!
 adam.battle_cry
 # => Adam says zing!!!

JavaScript

  Ninja.prototype.battleCry = function () {
   puts(this.name + " says zing!!!");
  }
  drew.battleCry();
  // => Drew says zing!!!
  adam.battleCry();
  // => Adam says zing!!!

Python

  def battle_cry(self):
   print '%s says zing!!!' % self.name
   Ninja.battle_cry = battle_cry
  drew.battle_cry()
  # => Drew says zing!!!
  adam.battle_cry()
  # => Adam says zing!!!

Invoking a method dynamically

The next thing we’ll look at is calling methods dynamically. The ability to call a method with a name given to us at runtime is a powerful tool in DSL creation and metaprogramming. Here’s the code. In case of Ruby, we supply the method name as a string and invoke it.

Ruby

 drew.send(:battle_cry)
 # => Drew says zing!!!

JavaScript

 drew['battleCry']();
 // => Drew says zing!!!

Python

 drew.__getattribute__('battle_cry')()
 # => Drew says zing!!!

Defining class level methods dynamically with closures

Here a class level method is defined which closes over some of the attributes in its context (in this case the method color is able to access the variable color_name as a closure). In Ruby, we’re using send to call define_method. This is a trick (hack) in ruby that allows us to call private methods without self being the implicit receiver. define_method takes a name for the new method and a block which describes the body of the method. In ruby, this block is a closure which means we have access to the scope at the time of the method definition. In Javascript example, Javascript functions are closures. This means we get the ability to capture the scope at the time of definition for free. The ubiquity of closures in Javascript is extremely powerful and, as we have seen so far, makes metaprogramming very easy.

Ruby

 color_name = 'black'
   Ninja.send(:define_method, 'color') do
   puts "#{name}'s color is #{color_name}"
 end
 drew.color
 # => Drew's color is black
 adam.color
 # => Adam's color is black

JavaScript

 var colorName = "black";
 Ninja.prototype['color'] = function () {
   puts(this.name + "'s color is " + colorName);
 }
 drew.color();
 // => Drew's color is black
 adam.color();
 // => Adam's color is black

Python

 color_name = 'black'
 def color(self):
   print "%s's color is %s" % (self.name, color_name)
 Ninja.color = color
 drew.color()
 # => Drew's color is black
 adam.color()
 # => Adam's color is black

Putting it all together

How does metaprogamming work with dynamic typed languages?

• Interpreted Languages

Most dynamic languages are interpreted, not compiled. The basic idea here is for the code to be interpreted at run-time, allows generic objects to dynamically interact with each other. This interaction in itself is potentially manipulating the behavior of the program as it is executing. Generic objects are not the only uniqueness of an interpreted language, the 'eval' statement is something else to consider. The ability to evaluate an expression via the 'eval' statement allows for the program to redefine functions at run-time. These two capabilities for dynamic languages have become immensely valuable to programmers.

• Dynamic methods

These types of methods are very flexible. The program have have an object created from a class, then you can add additional methods to the instance of that class. Therefore manipulating the program at run time by allowing for only this instance of the object to have additional methods. Not to be confused with extending classes or adding additional methods to classes.

• Singletons classes

Singleton classes should not be mistake for the Singleton design pattern. These types of classes available in some dynamic languages allow for dynamic dispatching. An common technique often used on singleton classes is to add dynamic methods to the instance.

• "Everything is an object"

Many languages a 'weak-typed' which means that potentially anything can be anything else. The most obvious way to have this behavior is by making everything an object. What historically are considered primitive data types now are all considered objects (e.g. string, integers, etc). In some of these languages even functions and procedures are considered objects. In some languages such as Ruby, even classes are Objects since they (as well as everything else) inherits from Object. Since classes can be used to define, extend, or redefine other classes (often referred to as a metaclass) this is also potentially changing the behavior of the original program.

Some other traits of some common dynamic languages worth mentioning are: the use of method_missing, introspection, lazy loading, declarative code, extensible type system, and method aliasing.

Advantages & Disadvantages

Advantages

• Using metaprogramming, you can create more expressive APIs (for example ActiveRecord uses metaprogramming to define accessor methods based on a table's column names, so you can write things like person.age instead of something like person.read_attribute("age"), where person is an active record object and the people table has a column called age) and you can accomplish some things with significantly less code than you otherwise would.

• Performance: Meta-programs represent data to be processed in terms of the low level data structures used by the script interpreter to represent the scripting language itself. Therefore, instead of being processed by an interpreted script manipulating script-level data structures, the data is processed by the compiled code of the language interpreter manipulating low level data structures directly. This removes the levels of indirection that slow down the execution of scripts, and so results in significant performance improvements.

Disadvanatges

• Metaprogramming code can be very difficult to read and understand: that is because the metaprogramming code will not necessarily express what the code it is creating is about, but only how it is creating that code. As such the code it is creating can be invisible to you as a reader of the source code - the created code will only start to exist at runtime. One would therefore expect that programmers would try especially hard when they metaprogram to make that particular kind of code expressive and easy to understand.

• Another consequence of the fact that the code produced by metaprogramming is not necessarily visible, is that debugging becomes more difficult: when analyzing problems you’ll not only be unsure how the programm works, but in addition, you won’t even be sure how the code that is executed looks like - since it is only generated at runtime.

Conclusion

Dynamic languages are increasingly recognized for playing a major role in software development, specifically for quick prototyping because of the forgiveness of the languages letting requirements evolve. These languages are historically looked at as scripting languages that are inexpensive and not scalable. Once you're familiar with the techniques, metaprogramming is not as complicated as it might sound initially. Metaprogramming allows you to automate error-prone or repetitive programming tasks. You can use it to pre-generate data tables, to generate boilerplate code automatically that can't be abstracted into a function, or even to test your ingenuity on writing self-replicating code.

References

Websites:

1. http://en.wikipedia.org/wiki/Metaprogramming
2. http://en.wikipedia.org/wiki/Dynamic_programming_language
3. http://en.wikipedia.org/wiki/Application_programming_interface
4. http://www.linuxjournal.com/article/9604
5. http://www.ibm.com/developerworks/linux/library/l-metaprog1/index.html
6. http://tratt.net/laurie/research/publications/html/tratt__dynamically_typed_languages/
7. http://www.sitepoint.com/typing-versus-dynamic-typing/
8. http://www.bias2build.com/thesis/ruby_v_js_MP.html
9. http://codeblog.dhananjaynene.com/2010/01/dynamically-adding-methods-with-metaprogramming-ruby-and-python/
10. http://fingernailsinoatmeal.com/post/292301859/metaprogramming-ruby-vs-javascript
11. http://www.ibm.com/developerworks/linux/library/l-pymeta/index.html

Books/Articles:

1. Tratt, L. 2005. Compile-time meta-programming in a dynamically typed OO language. In Proceedings of the 2005 Symposium on Dynamic Languages (San Diego, California, October 18 - 18, 2005). DLS '05. ACM, New York, NY, 49-63. http://doi.acm.org.www.lib.ncsu.edu:2048/10.1145/1146841.1146846
2. Tratt, L. 2009. Dynamically typed languages, Advances in Computers, 77, pp. 149-184
3. Perrotta, P. 2010. Metaprogramming Ruby: Program Like the Ruby Pros