CSC/ECE 517 Fall 2012/1w41 as

Introduction

In this page, the concept of meta-programming is discussed. We first give the definition of meta-programming, along with a brief description. Next, we give an overview of the concept of meta-program in general i.e. its evolution and implementation in various languages is discussed. We then focus on the languages in which Meta -programming is prevalent namely Ruby, Java, C, C++ and Python. We then follow it up with the advantages and disadvantages of the same before concluding.

Definition

Meta-programming is the discipline of writing programs that represent and manipulate other programs or themselves. An established example of meta-program is a compiler. We can also define a meta-program as a program which generates code and writing such programs is called meta-programming. This allows programmers to get more work done in the same amount of time as they would take to write all the code manually, or it gives programs greater flexibility to efficiently handle new situations without recompilation. The ability of a programming language to be its own meta-language is called reflection or reflexivity. This reflection ability is a feature that is very valuable to facilitate meta-programming.

Evolution of Meta-programming

Meta-programming was first introduced in Lisp. In Lisp, instead of just programming towards the language, we usually build the language up to our program. This is where the meta-programming comes into picture. For meta-programming purpose, Lisp provides macros as a standard facility to write code that will be directly be read and compiled by the system at run time.

For Meta programming in C-language,one of the popular examples is the Lex/Yacc system for generation of parsers and lexical scanners. That is, it reads an input stream specifying the lexical analyzer and outputs source code implementing the lexer in C.

For Meta programming in C++, it is done with C++ templates i.e., we define the program’s content by writing code that generates it. The code is generally written in between question marks.

Similar programs exist for Java and C#. The general idea, however, is that such an approach can be used anywhere we have a formal specification for which we know how to generate code.

Prolog, a generic purpose programming language, which uses the same data structures as the meta-programming and also provides simple clause expansions, is generally used for meta-programming.

How Meta-programming works

Meta-programming usually works in one of three ways. The first way is to expose the internals of the run-time engine to the programming code through application programming interfaces (APIs).

The second approach is dynamic execution of expressions that contain programming commands, often composed from strings, but can also be from other methods using arguments and/or context. Thus, "programs can write programs."

The third way is to step outside the language entirely. General purpose program transformation systems, which accept language descriptions and can carry out arbitrary transformations on those languages, are direct implementations of general meta-programming. This allows meta-programming to be applied to virtually any target language without regard to whether that target language has any meta-programming abilities of its own.

Uses of Meta-programming

Let us look at some of the uses of meta-programming:

• The major use of meta-programming is that it pre-generates the code at run-time. For example, if we are writing an application and we want a quick lookup table containing keys and values associated with the keys, we can have the program build the table at startup during runtime.

• It is the key ingredient for Domain Specific Languages. That is we use meta-programming to encapsulate domain-specific knowledge.

• Using meta-programming we can easily reduce the cost and time of development for a system by one order of magnitude, while improving the overall quality of the resulting code.

Meta programming in Ruby

Here we discuss about the implementation of meta-programming in a programming language called Ruby. We will also discuss about various techniques which help in implementing meta-programming. Ruby is an object-oriented and dynamic language. Ruby is also reflexive i.e., Ruby program can observe and modify its own structure and behavior at runtime. Also in Ruby, code is nothing but data and data is nothing but code. All these features of Ruby make it implement one of its most powerful features called meta-programming efficiently. In Ruby, with this meta-programming feature, we can write code that manipulates language constructs like instance variables, classes and modules at runtime. In Ruby, the classes (both standard classes and the classes created by us) are never closed i.e., we can always add extra methods to an existing class. All we need to do is open the class to which we want to add additional methods and then define the method we want to add. For instance, consider the following example:

  a = [1, 2, 3, 4]

Here ‘a’ is an array and if we want to find the sum of all the elements of the array, we need to open the Array class and add the method to calculate the sum of the array elements.

  class Array
     def sum
      inject{|a,x| a+x}
     end
  end

We can now call method product for the array.

      a.sum
      => 10

In Ruby, we can show as if there were two methods with same name. With this property, we can implement the concept of meta-programming efficiently in Ruby. Suppose, we have a class called Rectangle and it has an initialize method. Here, we can now instantiate this Rectangle by two ways. That is, either by passing in the coordinates of its corners or by passing in any of its corner along with the length and width of the corner. So, in this manner, even though there is only one initialize method, we can act as if there were two.

  # The Rectangle constructor accepts arguments in either
  # of the following forms:
  class Rectangle
     def initialize(*args)
        if args.size < 2  || args.size > 3
           puts 'Sorry wrong number of arguments passed. This method takes either 2 or 3 arguments.'  
        else
           puts 'Correct number of arguments passed!!!'
           if args.size == 2
              puts 'Two arguments'
           else
              puts 'Three arguments'
           end
        end
      end
  end
  Rectangle.new([10, 23], 4, 10)
  #=> Correct number of arguements passed!!!
  #=> Three arguements 
  Rectangle.new([10, 23], [14, 13])
  #=> Correct number of arguements passed!!!
  #=> Two arguements

Singleton classes

Singleton classes are one of the major properties of Ruby which make implementation of meta-programming efficient. Singleton class also called as meta-class or anonymous class is a class that acts as a proxy to the objects of a class. In Ruby, any object will have a class of which it is an instance (the class of the object can be found by calling the method class on it). Ruby has a feature where we can add additional methods to an object. So, when this is done, a new anonymous class is created which acts as an intermediate between the object and its actual class. This intermediate class or proxy class is called as a singleton class. We can consider the above example of array and finding the sum for this as well.

Method aliasing

Method aliasing is a technique used in Ruby to implement the concept of meta-programming. The technique that allows us to give aliases or new names to already existing variables, operators and methods is called as method aliasing. It can also be used to wrap an existing method effectively by intercepting any calls and injecting the desired behavior. Here the new reference or the aliasing name may not be a local, instance, constant, or class variable if we are aliasing a variable. Using this method or technique we can perform method over riding efficiently and thereby change the behavior of an object or a class. There are two keywords in Ruby which provide this method aliasing. They are alias and alias_method. The following is the syntax for both these keywords:

      alias        	:new  :old
      alias_method     :new, :old

Generally we use alias to give a second name to a method or variable of a class, whereas we use alias_method when we want to assign second name to a method in a Module. So basically, alias keyword takes two arguments as we can see in the syntax. The first argument is the new name or the second name and second argument is the old name or the original name. Let us look at an example that illustrates this technique. Consider the following example:

   class Array
    def sum
        inject {|a, x| a+x }
    end
    alias  :newsum:sum
   end
   a = Array.new([1, 2, 3, 4])
   a.newsum   #=> 10

In this example, we are defining a new method called sum and then aliasing that method with a new name called newsum in the class Array. Now we can call this method with its alias name as above and still the method will be invoked. This is how method aliasing can be implementing. We can notice here that, this technique can be used to add new methods and alias those methods for a given class. In this manner we can also change the behavior of the class. Also, we can call the aliasing name of a method inside another method thereby implementing method over riding concept. Consider the following example to illustrate this:

        class A
         def method1
            puts ”This is the main method”
          end
          alias :method2 :method1
          def method1
             puts “This is not main method"
             method2
          end
        end
        a = A.new()
        a.method1
        => This is not main method
        => This is main method	

So, in this manner method over riding is done using method aliasing. We can thus say that method aliasing can be used to implement the concept of meta-programming.

Meta-programming in C++

The need for representing meta-information at compile time became apparent during the development of the C++ Standard library. The problems were solved by the traits template idiom. The first article on template meta-programming was published in 1995. The IF<> template was the first control structure in a generic form.

Support for meta-programming

The meta-programming support in C++ is not a designed feature but more or less abuse of C++ compiler. The code of meta-programs is quite peculiar and obscure. There are also limitations for complexity of the meta-programs set by the compilers.

Reflection

Different languages provide different levels of reflection. While Smalltalk allows you to directly modify classes or methods by modifying their meta-objects at runtime, Java provides a much lower lever of reflection, mainly suitable for JavaBeans component model and the Remote Method Invocation mechanism. C++ provides even less support for reflection than Java, the most important feature being runtime type-identification (RTTI). There are also reflective extensions available to C++

Two-Level Languages

An important concept to static meta-programming is a concept of two-level languages. Two-level languages contain static code, which is evaluated at compile time, and dynamic code, which is compiled and later executed at runtime. ISO/ANSI C++ contains a template mechanism for defining parameterized classes and functions. This includes type and integral template parameters and partial ad full template specialization. Templates together with other C++ features constitute Turing-complete, compile time sub-language of C++ (and so C++ is a two-level language). Because the sub-language is Turing complete, there are no theoretical limits to what you can implement with it. In practice, there might be technical limitations, such as compiler limits. The main compile-time conditional construct is template specialization: The compiler has to select a matching template out of several alternatives. The compile-time looping construct is template recursion (e.g. a member of class templates used in its own definition.). The compiler has to expand such patterns recursively.

Example

Factorial- One of the most common examples of using static code (i.e., code executed at compile time) is computing the factorial n! = 1. 2. 3... (n - 1). n of a natural number. A conventional C++ factorial function is as follows

 int factorial(int n)
 {
 return (n==0)? 1: n*factorial(n-1);
 }

The corresponding static code for computing the factorial at compile time is given as follows; RET is used as an abbreviation for a return statement of a conventional function. When the compiler tries to instantiate the structure template Factorial<7> (where n=7). This involves initializing the enumerator RET with Factorial<6>:: RET*7 and the compiler has to instantiate Factorial<6>::RET (and so on). The template specialization matches for n=0 (i.e. Factorial<0>).

 template<int n>
 struct Factorial
 { 
 enum {RET=Factorial<n-1>::RET*n};
 };
 // Note: template specialization
 template<>
 struct Factorial<0>
 { 
 enum{RET=1};
 };
 // Usage:
 cout << "Factorial(7)= " << Factoral<7>::RET << endl;

Meta-programming in JAVA

Java meta-programming mechanisms:

• Reflection

• Generics

• Metadata annotations

Reflection

Reflection is the ability of a program to manipulate as data something representing the state of the program during its own execution. Usually this meta-level information is modeled using the general abstraction mechanisms available in the language. Java's Reflection API's makes it possible to inspect classes, interfaces, fields and methods at runtime, without knowing the names of the classes, methods etc. at compile time. It is also possible to instantiate new objects, invoke methods and get/set field values using reflection.

The following is an example in Java using the Java package java.lang.reflect:

// Without reflection
new Foo().hello();

// With reflection
Class<?> clazz = Class.forName("Foo");
clazz.getMethod("hello").invoke(clazz.newInstance());

Uses of Reflection

Reflection is commonly used by programs which require the ability to examine or modify the runtime behavior of applications running in the Java virtual machine. This is a relatively advanced feature and should be used only by developers who have a strong grasp of the fundamentals of the language.

Extensibility Features

An application may make use of external, user-defined classes by creating instances of extensibility objects using their fully-qualified names.

Class Browsers and Visual Development Environments

A class browser needs to be able to enumerate the members of classes. Visual development environments can benefit from making use of type information available in reflection to aid the developer in writing correct code.

Debuggers and Test Tools

Debuggers need to be able to examine private members on classes. Test harnesses can make use of reflection to systematically call a discoverable set APIs defined on a class, to insure a high level of code coverage in a test suite.

Drawbacks of Reflection

Reflection is powerful, but should not be used indiscriminately. If it is possible to perform an operation without using reflection, then it is preferable to avoid using it. The following concerns should be kept in mind when accessing code via reflection.

• Performance Overhead

Because reflection involves types that are dynamically resolved, certain Java virtual machine optimizations cannot be performed. Consequently, reflective operations have slower performance than their non-reflective counterparts, and should be avoided in sections of code which are called frequently in performance-sensitive applications.

• Security Restrictions

Reflection requires a runtime permission which may not be present when running under a security manager. This is in an important consideration for code which has to run in a restricted security context, such as in an Applet.

• Exposure of Internals

Since reflection allows code to perform operations that would be illegal in non-reflective code, such as accessing private fields and methods the use of reflection can result in unexpected side-effects, which may render code dysfunctional and may destroy portability. Reflective code breaks abstractions and therefore may change behavior with upgrades of the platform.

Generics

Generic in Java is added to provide compile time type-safety of code and removing risk of ClassCastException at runtime which was quite frequent error in Java code. Generics allows Java programmer to write more robust and type-safe code.

Hierarchy and classification

As per Java Language Specification:

• A type variable is an unqualified identifier. Type variables are introduced by generic class declarations, generic interface declarations, generic method declarations, and by generic constructor declarations.

• A class is generic if it declares one or more type variables. These type variables are known as the type parameters of the class. It defines one or more type variables that act as parameters. A generic class declaration defines a set of parameterized types, one for each possible invocation of the type parameter section. All of these parameterized types share the same class at runtime.

• An interface is generic if it declares one or more type variables. These type variables are known as the type parameters of the interface. It defines one or more type variables that act as parameters. A generic interface declaration defines a set of types, one for each possible invocation of the type parameter section. All parameterized types share the same interface at runtime.

• A method is generic if it declares one or more type variables. These type variables are known as the formal type parameters of the method. The form of the formal type parameter list is identical to a type parameter list of a class or interface.

• A constructor can be declared as generic, independently of whether the class that the constructor is declared in is itself generic. A constructor is generic if it declares one or more type variables. These type variables are known as the formal type parameters of the constructor. The form of the formal type parameter list is identical to a type parameter list of a generic class or interface, the interface constructor is generic.

The following block of Java code illustrates a problem that exists when not using generics. First, it declares an ArrayList of type Object. Then, it adds a String to the ArrayList. Finally, it attempts to retrieve the added String and cast it to an Integer.

 List v = new ArrayList();
 v.add("test");
 Integer i = (Integer)v.get(0);        // Run time error

Although the code compiles without error, it throws a runtime exception (java.lang.ClassCastException) when executing the third line of code. This type of problem can be avoided by using generics and is the primary motivation for using generics.

Using generics, the above code fragment can be rewritten as follows:

 List<String> v = new ArrayList<String>();
 v.add("test");
 Integer i = v.get(0); // (type error)  Compile time error

The type parameter String within the angle brackets declares the ArrayList to be constituted of String (a descendant of the ArrayList's generic Object constituents). With generics, it is no longer necessary to cast the third line to any particular type, because the result of v.get(0) is defined as String by the code generated by the compiler.

Generic Class Definitions

Here is an example of a generic class:

public class Entry<K, V> {

 private final K key;
 private final V value;

 public Entry(K k,V v) {  
   key = k;
   value = v;   
 }

 public K getKey() {
   return key;
 }

 public V getValue() {
   return value;
 }

 public String toString() { 
   return "(" + key + ", " + value + ")";  
 }
}

This generic class can be used in the following way:

Entry<String, String> grade440 = new Entry<String, String>("mike", "A");
Entry<String, Integer> marks440 = new Entry<String, Integer>("mike", 100);
System.out.println("grade: " + grade440);
System.out.println("marks: " + marks440);

Generic method definitions

Here is an example of a generic method using the generic class above:

 public static <T> Entry<T,T> twice(T value) {
  return new SimpleImmutableEntry<T,T>(value, value);
 }

In many cases the user of the method need not indicate the type parameters, as they can be inferred:

Entry<String, String> pair = twice("Hello");

The parameters can be explicitly added if needed:

Entry<String, String> pair = this.<String>twice("Hello");

Generic methods can be defined based on given parameters.

public <T> T[] toArray(T... elements) {
  return elements;
}

Drawbacks of Generics

• At run-time, all generic type information is lost.

• At run-time, all type parameter instances are converted to/from Object references, meaning that there is a high upcast/downcast overhead when using generics.

• All instances of generic classes have the exact same static members.

• It is not possible to throw generic exceptions (generic classes derived from Throwable).

Metadata Annotations

An annotation, in the Java computer programming language, is a special form of syntactic metadata that can be added to Java source code. Classes, methods, variables, parameters and packages may be annotated. Java annotations can be reflective in that they can be embedded in class files generated by the compiler and may be retained by the Java VM to be made retrievable at run-time.

Built-In Annotations

Java defines a set of annotations that are built into the language.

Annotations applied to java code

• @Override - Checks that the function is an override. Causes a compile warning if the function is not found in one of the parent classes.

• @Deprecated - Marks the function as obsolete. Causes a compile warning if the function is used.

• @SuppressWarnings - Instructs the compiler to suppress the compile time warnings specified in the annotation parameters

This following example shows the use of the @Override annotation. It instructs the compiler to check parent classes for matching functions. In this case, an error is generated that the Cat's gettype() function does not in fact override Animal's getType() as desired, but instead declares a new function due to the name mismatch.

public class Animal {

   public void speak() {
   }

   public String getType() {
      return "Generic animal";
   }
}
 public class Cat extends Animal {

   @Override
   public void speak() { // This is a good override.
      System.out.println("Meow."); 
   }

   @Override
   public String gettype() { // throws compile warning due to mistyped name.
      return "Cat";
   }
}

Metaprogramming: Benefits/Drawbacks

Benefits:

• Simplicity--The biggest benefit of meta-programming is that from a code-base perspective, it keeps things really simple. Because no code is generated, your classes remain incredibly small. For the most part, the only code that exists in your classes is custom methods to handle specific business rules and other specific one-off customizations.And in fact, code generation can sometimes be quite daunting for people simply because the generated code increases the size of the codebase, and thus, it would at least feel like the system would be a bit more unwieldy to manage.

• Real-time Changes--Because the analysis is done at runtime, changes to your data model can be reflected in your application at real-time. There is no need to “regenerate” your code as you would with a code generation approach. But of course, this runtime analysis does incur a cost.

• Metaprogramming can accomplish truly generic code, facilitating code minimization and better maintainability.

• Meta-programming leads to better functionality.

• Meta-programming provides less effort for programmers and reduces the maintenance effort.

• Meta code is generated at compile time. Thus, results faster programs.

Drawbacks

• Impossible to debug.

• Hard to trace.

• Ambiguous to some extent.

• Run-time reflection reduces performance.

• Compilation may become slow.

• Metaprograms can be difficult to maintain by programmers inexperienced in metaprogramming.

References

1. http://blog.grahampoulter.com/2008/12/code-generation-vs-metaprogramming-in.html

2. http://www.qcodo.com/wiki/article/background/metaprogramming

3. http://code-jam.blogspot.com/2006/07/metaprogramming.html

4. http://en.wikipedia.org/wiki/Reflection_%28computer_programming%29

5. http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/package-summary.html

6. http://en.wikipedia.org/wiki/Generics_in_Java

7. http://tutorials.jenkov.com/java-generics/index.html

8. http://www.tutorialspoint.com/java/java_generics.htm

9. http://javarevisited.blogspot.com/2011/09/generics-java-example-tutorial.html

10. http://en.wikipedia.org/wiki/Java_annotation

11. http://docs.oracle.com/javaee/6/tutorial/doc/girdd.html

12. http://www.cs.tut.fi/~kk/webstuff/MetaprogrammingCpp.pdf

13. http://www.ciaranmchale.com/download/java-reflection-explained-simply-manual-8up.pdf

14. http://tutorials.jenkov.com/java-reflection/classes.html

15. http://webcourse.cs.technion.ac.il/236703/Spring2012/ho/WCFiles/7b%20-%20Reflection%20in%20Java.pdf

16. http://docs.oracle.com/javase/tutorial/reflect/index.html

17. http://en.wikipedia.org/wiki/Metaprogramming

18. http://media.pragprog.com/titles/ppmetr/introduction.pdf

19. http://www.vanderburg.org/Speaking/Stuff/oscon05.pdf

20. http://www.ibm.com/developerworks/linux/library/l-metaprog1/index.html

21. http://en.wikipedia.org/wiki/Metaprogramming

22. http://www.cs.tut.fi/~kk/webstuff/MetaProgrammingJavaKalvot.pdf

Further Reading

1. http://c2.com/cgi/wiki?MetaProgramming

2. http://rubylearning.com/blog/2010/11/23/dont-know-metaprogramming-in-ruby/

3. http://rubysource.com/ruby-metaprogramming-part-i/