CSC/ECE 517 Fall 2010/ch2 S24 rm

"I'd rather write programs that write programs than write programs" - Richard Sites

Introduction

Most metaprogramming is done in dynamic languages, like Ruby. Achieving metaprogramming in static languages directly, becomes complex due to its inherent nature of compile time abstraction verification. However, there are tools and packages for support of metaprogramming in statically typed languages, such as Java that can be leveraged to achieve metaprogramming features.

Metaprogramming

Metaprogramming is a programming technique of writing computer programs that write or manipulate other programs or themselves. In other words, it is a programming technique of writing programs with a higher level of abstraction to make it appear as generative programming.

Metaprogramming involves two kinds of languages. The meta-language is the language in which meta-programs, which construct or manipulate other programs, are written. The object-language is the language of programs being manipulated. This makes them ‘meta level programs’ whose problem domain are other ‘base level programs’.[1]

The ability of a programming language to be its own metalanguage is called reflection or reflexivity.

Simple example of a metaprogram: Let us consider a totally fabricated example for our understanding at very high level. Suppose we need to write a C program that printed the following 500 lines of text with a restriction that the program could not use any kind of loop or goto instruction.

Output expected:

 1 Mississippi
 2 Mississippi
 3 Mississippi
 4 Mississippi
 ...
 499 Mississippi
 500 Mississippi

In C this would be then coded as:

 #include <stdio.h>
 int main(void) {
   printf("1 Mississippi\n");
   printf("2 Mississippi\n");
       -
       -
       -
   printf("499 Mississippi\n");
   printf("500 Mississippi\n");
   return 0;
  }

With the power of a metaprogramming language we can write another program that writes this program automatically.

Ruby code:

 File.open('mississippi.c', 'w') do |output|
  output.puts '#include <stdio.h>'
  output.puts 'int main(void) {'
    1.upto(500) do |i|
      output.puts "    printf(\"#{i} " +
      "Mississippi\\n\");"
  end
  output.puts '    return 0;'
  output.puts '}'
 end

This code creates a file called mississippi.c with the expected 500+ lines of C source code.Here, mississippi.c is the generated code and ruby code is the metaprogram.

Applications of Metaprogramming

Metaprogramming is an attractive technique needed when one needs to alter the behavior of a program at run time. Due to its generative nature, it has numerous applications in program development. It can achieve program development without rewriting boiler-plate code all the time, ensuring efficiency, increasing modularity and minimizing inconsistent implementation errors. Program generators and program analyzers are the two main categories of meta programs. Metaprograms can be compilers, interpreters, type checkers etc. Some commonly used applications include using a program that outputs source code to -

• generate sine/cosine/whatever lookup tables
• to extract a source-form representation of a binary file
• to compile your bitmaps into fast display routines
• to extract documentation, initialization/finalization code, description tables, as well as normal code from the same source files
• to have customized assembly code, generated from a perl/shell/scheme script that does arbitrary processing
• to propagate data defined at one point only into several cross-referencing tables and code chunks. [33]

In many cases, this allows programmers to get more done in the same amount of time as they would take to write all the code manually, or it gives programs greater flexibility to efficiently handle new situations without recompilation. [1]

Typing in Programming Languages

Earlier programming languages [e.g. Assembly] were written such that each machine level function was reflected in the program code. With advancement in programming languages a certain level of abstraction was reached wherein lower level details were abstracted with one functional unit of work and represented by fewer lines of code e.g. primitive variables are represented with higher level abstract classes. With this abstraction arose a need for checking the validity of operations that could be performed with these abstractions in place.

Typing in programming languages is property of operations and variables in the language that ensure that certain kinds of values that are invalid are not used in operations with each other. Errors related to these are known as type errors. Type checking is the process of verifying and enforcing the constraints of types. Compile time type checking also known as static type checking. Run time type checking is known as dynamic type checking. If a language specification requires its typing rules strongly (i.e., more or less allowing only those automatic type conversions which do not lose information), one can refer to the process as strongly typed, if not, as weakly typed.[8] The above classification can be represented as -

Statically Typed Programming Languages

Statically typed languages ensure that a fixed type is assigned by the programmer to every variable and parameter. Thus, every expression type can be deduced and type checked during compilation. Static languages try to fix most errors during compile time and strive to minimize failures during run time. Due to this there are many type constraints on the programmer while coding. At run time, the program uses the classes that it has been given and in this way statically typed languages make distinctions between what happens at compile time and what happens at run time. Examples of statically typed languages are C, C++, Java, C#.

Dynamically Typed Programming Languages

In dynamically typed languages, the variables and parameters do not have a designated type and may take different values at different times. In all the operations, the operands must be type checked at runtime just before performing the operation. Dynamically typed languages don’t need to make a distinction between classes created at compile time and classes provided. It is possible to define classes at run time and in fact, classes are always defined at run time. These eliminate many developer constraints by avoiding the need of book keeping, declarations etc. Due to this flexibility these languages make an ideal candidate for prototyping and are widely used in agile development environments. However, dynamic languages are known to have performance issues. Static languages have code optimization features at compile time, but dynamic languages allow runtime code optimizations only. [7] In dynamically typed languages, the interpreter deduces type and type conversions, this makes development time faster, but it also can provoke runtime failures. These runtime failures are caught early on during compile time for statically typed languages. Examples of dynamically typed languages include Perl, Python, JavaScript, PHP, Ruby, Groovy.

Metaprogramming in statically typed languages

In safety languages [syntactically verbose], metaprogramming is not a standard feature, it can however be achieved. Also, static typing in meta-programs has a number of advantages. In addition to guaranteeing that the meta-program encounters no type-errors while manipulating object-programs, a statically typed metaprogramming language can also guarantee that any of the object-programs generated by the meta-program are also type-correct. A disadvantage of these type system is that (in case of meta-programming languages with weaker type systems) they sometime may be too restrictive in object-programs that the programmer is allowed to construct.

Techniques and Packages

Many language features can be leveraged to achieve some form of characteristics needed to achieve metaprogramming. For instance languages that support reflection also allow for dynamic code generation. e.g. In Microsoft .NET Framework use of System.Reflection.Emit namespace is used to generate types and methods at runtime. [12]

Reflection

Reflection is a valuable language feature to facilitate metaprogramming. Reflection is defined as the ability of a programming language to be its own meta-language. Thus, reflection is writing programs that manipulate other programs or themselves. e.g. In Java, reflection enables to discover information about the loaded classes:

• Fields,
• Methods and constructors
• Generics information
• Metadata annotations

It also enables to use these metaobjects to their instances in run time environment. E.g. Method.invoke(Object o, Object… args) With the Java reflection API, you can interrogate an object to get all sorts of information about its class. [14]

Consider the following simple example:

 public class HelloWorld {
   public void printName() {
     System.out.println(this.getClass().getName());
   }
 }

The line

 (new HelloWorld()).printName();

sends the string HelloWorld to standard out. Now let x be an instance of HelloWorld or one of its subclasses. The line

 x.printName();

sends the string naming the class to standard out.

The printName method examines the object for its class (this.getClass()). In doing so, the decision of what to print is made by delegating to the object's class. The method acts on this decision by printing the returned name. Without being overridden, the printName method behaves differently for each subclass than it does for HelloWorld. The printName method is flexible; it adapts to the class that inherits it, causing the change in behavior. Simple example to attain flexibility using reflection. [15 example]

Annotations

Annotations are a metaprogramming facility that allow the code to be marked with defined tags. Many APIs require a fair amount of boilerplate code ie. code that can be reused in new contexts or applications without being changed much from the original. This boilerplate could be generated automatically by a tool if the program were “decorated” with annotations indicating which methods were remotely accessible. Metadata provided using annotations is beneficial for documentation, compiler checking, and code analysis. One can use this metadata to indicate if methods are dependent on other methods, if they are incomplete, if a certain class must reference another class, and so on. It is used by the compiler to perform some basic compile-time checking. For example there is a override annotation that lets you specify that a method overrides another method from a superclass. At this, the Java compiler will ensure that the behavior you indicate in your metadata actually happens at a code level as well.

An “annotation” has an “annotation type” associated with it which is used for defining it. It is used when you want to create a custom annotation. The type is the actual construct used, and the annotation is the specific usage of that type. An annotation type definition takes an "at" (@) sign, followed by the interface keyword plus the annotation name. On the other hand, an annotation takes the form of an "at" sign (@), followed by the annotation type [23].

Example to Define an Annotation (Annotation type)

 public @interface MyAnnotation {
   String doSomething();}
  Example to Annotate Your Code (Annotation)
  MyAnnotation (doSomething="What to do")
  public void mymethod() {
  ....
  }

Annotation Types

There are three annotation types:

• Marker: Marker type annotations have no elements, except the annotation name itself.

Example:

 public @interface MyAnnotation {
 } 

Usage:

 @MyAnnotation
 public void mymethod() {
  ....
 }

• Single-Element: Single-element, or single-value type, annotations provide a single piece of data only. This can be represented with a data=value pair or, simply with the value (a shortcut syntax) only, within parenthesis.

Example:

 public @interface MyAnnotation
 {
   String doSomething();
 } 

Usage:

 @MyAnnotation ("What to do")
 public void mymethod() {
  ....
 } 

3. Full-value or multi-value: Full-value type annotations have multiple data members. Therefore, you must use a full data=value parameter syntax for each member. Example:

 public @interface MyAnnotation {
   String doSomething();
   int count; String date();
 } 

Usage:

 @MyAnnotation (doSomething="What to do", count=1,
             date="09-09-2005")
 public void mymethod() {

Generics

Generic programming is a style of computer programming in which algorithms are written in terms of to-be-specified-later types that are then instantiated when needed for specific types provided as parameters [21]. Java Generics are primarily a way for library authors to write something once, which users can customize to their own types. Generics allow the creation of classes and methods that work in the same way on different types of objects. The term "generic" comes from the idea that we'd like to be able to write general algorithms that can be broadly reused for many types of objects rather than having to adapt our code to fit each circumstance.

Generics add a way to specify concrete types to general purpose classes and methods that operated on object before. A Java collection is a flexible data structure that can hold heterogeneous objects where the elements may have any reference type. When you take an element out of a collection, you must cast it to the type of element that is stored in the collection. Generics provides a way for you to communicate the type of a collection to the compiler, so that it can be checked. Once the compiler knows the element type of the collection, the compiler can check that the collection has been used consistently and can insert the correct casts on values being taken out of the collection[22]. Generics are implemented by type erasure: generic type information is present only at compile time, after which it is erased by the compiler. The main advantage of this approach is that it provides total interoperability between generic code and legacy code that uses non-parameterized types (which are technically known as raw types).

Consider a non-generic example:

// Removes 4-letter words from c. Elements must be strings

  static void expurgate(Collection c)  
  {    
       for (Iterator i = c.iterator(); i.hasNext(); )      
       if (((String) i.next()).length() == 4)        
       i.remove();
   }

Here is the same example modified to use generics:

// Removes the 4-letter words from c

 static void expurgate(Collection<String> c) 
 {    
   for (Iterator<String> i = c.iterator(); i.hasNext(); )
   if (i.next().length() == 4)
   i.remove();
 }

The declaration above reads as “Collection of String c.” Collection is a generic class that takes ‘String’ as its type parameter. The code using generics is clearer and safer. Unsafe cast and a number of extra parentheses have been eliminated. More importantly, we have moved part of the specification of the method from a comment to its signature, so the compiler can verify at compile time that the type constraints are not violated at run time. [13]

Template Metaprogramming

Template Metaprogramming is a generic programming technique that uses extremely early binding. A primary requirement for a metaprogramming language is providing high-level abstractions to hide the internal representation of base programs. Each template language is specific for a base language and is generated from it. In this sense, a language of templates is a superset of the base language. Thus templates are abstractions that encapsulate a program pattern written by example. This concept has been explained in detail in section XXXX