CSC/ECE 517 Fall 2009/wiki2 1 SA: Difference between revisions
(32 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
= <center>'''Metaprogramming'''</center> = | = <center>'''Metaprogramming'''</center> = | ||
Line 5: | Line 4: | ||
== Introduction == | == Introduction == | ||
The term 'Meta' is just a preposition meaning “after”, “beyond”, or simply “with” – specially it means a higher level of abstraction, especially a self-referential abstraction. That’s the sense in which it is used in the term metaprogramming – modifying programs programmatically, or modifying the programming language itself. | |||
Metaprogramming, is the creation of procedures and programs that automatically construct the definitions of other procedures and programs. Metaprogramming automates some of the tedious and error-prone parts of the programmer's job. It refers to a variety of ways a program has knowledge of itself or can manipulate itself. It's basically "writing code that writes code". | |||
To understand metaprogramming, compiler is a very good example. It takes as input, code in some language, and creates code for another language. | To understand metaprogramming, compiler is a very good example. It takes as input, code in some language, and creates code for another language. | ||
The most liberal definition of metaprogramming is: | The most liberal definition of metaprogramming is: | ||
* Compile code generation or Runtime code generation (or both) | * Compile code generation or Runtime code generation (or both) | ||
* Aspect-Oriented Thinking or Aspect Oriented Programming | * Aspect-Oriented Thinking or [http://en.wikipedia.org/wiki/Aspect_Oriented_Programming Aspect Oriented Programming] | ||
* DRY Thinking | * [http://www.artima.com/weblogs/viewpost.jsp?thread=260578 DRY Thinking] i.e 'don't repeat yourself'. So when you find any duplication of code write a metaprogram to apply 'DRY' | ||
* [http://en.wikipedia.org/wiki/Template_metaprogramming Template metaprogramming] | |||
* Self modifying code | |||
* Self code generation | |||
== | == How Meta programming is implemented ? == | ||
There are many languages that do metaprogramming. In languages like C#, reflection is a form of metaprogramming since the program can examine information about itself. For example, returning a list of all the properties of an object. In languages like ActionScript, we can evaluate functions at runtime to create new programs such as eval("x" + i).DoSomething() would affect an object called x1 when i is 1 and x2 when i is 2. | There are many languages that do metaprogramming. In languages like C#, reflection is a form of metaprogramming since the program can examine information about itself. For example, returning a list of all the properties of an object. In languages like ActionScript, we can evaluate functions at runtime to create new programs such as eval("x" + i).DoSomething() would affect an object called x1 when i is 1 and x2 when i is 2. | ||
Line 21: | Line 24: | ||
It can be implemented by using any of these and in combination: | It can be implemented by using any of these and in combination: | ||
* Reflection | * [http://en.wikipedia.org/wiki/Reflection_%28computer_science%29 Reflection] | ||
* DSLs (Domain Specific Languages) | * [http://en.wikipedia.org/wiki/Domain_Specific_Language DSLs] (Domain Specific Languages) | ||
* (.NET) or Annotations (Java) | * (.NET) or Annotations (Java) | ||
* Generics (.NET/Java) | * [http://en.wikipedia.org/wiki/Generic_programming Generics] (.NET/Java) | ||
* Templates (C++) | * [http://en.wikipedia.org/wiki/Template_metaprogramming Templates] (C++, Haskell) | ||
* Method_missing (Ruby) | * Method_missing (Ruby) | ||
* Closures / first class functions / delegates | * [http://en.wikipedia.org/wiki/Closure_%28computer_science%29 Closures] / first class functions / delegates | ||
* AOP - Aspect Oriented Programming | * [http://en.wikipedia.org/wiki/Aspect_Oriented_Programming AOP] - Aspect Oriented Programming | ||
== Different languages that supports Metaprogramming == | == Different languages that supports Metaprogramming == | ||
Line 34: | Line 37: | ||
* Curl | * Curl | ||
* D | * D | ||
* Groovy | * Groovy | ||
* Haskell | * Haskell | ||
* Lisp | * Lisp | ||
* Maude system | * Maude system | ||
* MetaL | * MetaL | ||
* Nemerle | * Nemerle | ||
* Perl | * Perl | ||
Line 47: | Line 47: | ||
* Ruby | * Ruby | ||
* Smalltalk | * Smalltalk | ||
* C++ | * C++ | ||
Line 104: | Line 102: | ||
mixin(fooToD(import("example.foo"))); | mixin(fooToD(import("example.foo"))); | ||
} | } | ||
=== Metaprogramming in Groovy === | === Metaprogramming in Groovy === | ||
Groovy is an object oriented programming language for Java platform with dynamic language features. It is also used for scripting. It uses metaprogramming in the following way. | Groovy is an object oriented programming language for Java platform with dynamic language features. It is also used for scripting. Groovy has extensive metaprogramming facilities and one such addition is the ExpandoMetaClass, which enables us to do a host of copasetic things, including adding methods to normal Java objects (like String). It uses metaprogramming in the following way. | ||
Below example demonstrates using classes from the Jakarta Commons Lang package for metaprogramming. All of the methods in org.apache.commons.lang.StringUtils coincidentally follow the Category pattern — static methods that accept a String as the first parameter. This means that we can use the StringUtils class right out of the box as a Category.[http://www.ibm.com/developerworks/java/library/j-pg06239.html 1] | Below example demonstrates using classes from the Jakarta Commons Lang package for metaprogramming. All of the methods in org.apache.commons.lang.StringUtils coincidentally follow the Category pattern — static methods that accept a String as the first parameter. This means that we can use the StringUtils class right out of the box as a Category.[http://www.ibm.com/developerworks/java/library/j-pg06239.html 1] | ||
Line 187: | Line 181: | ||
~(head (tail priority)) endl) | ~(head (tail priority)) endl) | ||
(print stdout tab "Note: " ~note endl endl))) | (print stdout tab "Note: " ~note endl endl))) | ||
=== Metaprogramming in Maude System === | |||
Maude system is an implementation of rewriting logic and an heavy emphasis on reflection based metaprogramming. The following example illustrates it using a set of rewrite rules using reflection. Here NAT represents the set of natural numbers. s(0) indicates successor of 0 i.e. 1. the rewriting rules r1 are created using the reflection of itself. | |||
*'''Example:''' [http://en.wikipedia.org/wiki/Maude_system 17] | |||
*'''Example:''' [http://en.wikipedia.org/wiki/Maude_system | |||
<pre> | <pre> | ||
mod PERSON is | mod PERSON is | ||
including NAT . *** | including NAT . *** NAT is the set of natural numbers | ||
sort Person . | sort Person . | ||
sort State . | sort State . | ||
op married : -> State [ctor] . | op married : -> State [ctor] . | ||
op divorced : -> State [ctor] . | op divorced : -> State [ctor] . | ||
Line 209: | Line 202: | ||
op dead : -> State [ctor] . | op dead : -> State [ctor] . | ||
op person : State Nat -> Person [ctor] . | op person : State Nat -> Person [ctor] . | ||
var N : Nat . | var N : Nat . | ||
var S : State . | var S : State . | ||
rl [birthday] : | rl [birthday] : | ||
person (S, N) => person (S, N + s(0)) . | |||
person (S, N) => person (S, N + s(0)) . | rl [get-engaged] : | ||
person (single, N) => person (engaged, N) . | |||
rl [get-engaged] : | rl [get-married] : | ||
person (engaged, N) => person (married, N) . | |||
person (single, N) => person (engaged, N) . | rl [get-divorced] : | ||
person (married, N) => person (divorced, N) . | |||
rl [get-married] : | rl [las-vegas] : | ||
person (S, N) => person (married, N) . | |||
person (engaged, N) => person (married, N) . | rl [die] : | ||
person (S, N) => person (dead, N) . | |||
rl [get-divorced] : | endm | ||
</pre> | |||
person | In the above example a person still is the same person after he is marriedhis marital status has been changed. This is shown in the rewrite rules. These rules do not have to be confluent and terminating so it does matter a great deal what rules are chosen to rewrite the term. The rules are applied at 'random' by the Maude system, indicating no surity that one rule is applied before another rule and so on. | ||
=== Metaprogramming in MetaL === | === Metaprogramming in MetaL === | ||
MetaL is shorthand for Meta-programming Language. MetaL programs source code is based on XML. MetaL compiler engine can be used to generate the same program from MetaL source code to potentially any target language. Currently supported target languages are PHP, Java and Perl. | MetaL is shorthand for Meta-programming Language. MetaL programs source code is based on XML. MetaL compiler engine can be used to generate the same program from MetaL source code to potentially any target language. Currently supported target languages are PHP, Java and Perl. | ||
<br>The MetaL compiler engine traverses XML source code and executes actions associated with each command tag. Actions may be immediate, like creating a file, or generating some output, like the lines of code of a program in a given target language.Command actions are implemented by compiler modules. Such modules implement the different kinds of commands of the language, like execution flow, variable and expression manipulation, object oriented programming classes, etc.. Each module is responsible for generating the code in each of the supported target languages | <br>The MetaL compiler engine traverses XML source code and executes actions associated with each command tag. Actions may be immediate, like creating a file, or generating some output, like the lines of code of a program in a given target language.Command actions are implemented by compiler modules. Such modules implement the different kinds of commands of the language, like execution flow, variable and expression manipulation, object oriented programming classes, etc.. Each module is responsible for generating the code in each of the supported target languages | ||
Source code of the Hello world! program written in MetaL. | |||
<?xml version="1.0"?> | |||
<!-- | |||
@(#) $Id: sample.html,v 1.7 2006/01/12 19:51:37 mlemos Exp $ | |||
--> | |||
<output> | |||
<script> | |||
<string>Hello World! | |||
</string> | |||
</script> | |||
</output> | |||
=== Metaprogramming in Nemerle === | === Metaprogramming in Nemerle === | ||
Line 444: | Line 359: | ||
=== Metaprogramming in Ruby === | === Metaprogramming in Ruby === | ||
Ruby is a dynamically-typed, interpreted and object oriented language. Ruby is by default a metaprogramming language as the syntaxes if Ruby support that. Ruby has the following features which makes Ruby a metaprogramming language. | |||
* Dynamic and reflective | |||
* Everything is open to change | |||
* Blocks allow writing new control structures | |||
* Add continuations to really get fancy | |||
* Most declarations are executable statements | |||
Metaprograms in Ruby are written by using eval function, or using macro or using DSL. | |||
*'''''Example:''' | |||
We have a C program that needs to include a PNG image, but for some reason, the deployment platform can accept one file only, the executable file. Thus, the data that conforms the PNG file data has to be integrated within the program code itself. To achieve this, we can read the PNG file beforehand and generate the C source text for an array declaration, initialized with the corresponding data as literal values. This Ruby script does exactly that: | We have a C program that needs to include a PNG image, but for some reason, the deployment platform can accept one file only, the executable file. Thus, the data that conforms the PNG file data has to be integrated within the program code itself. To achieve this, we can read the PNG file beforehand and generate the C source text for an array declaration, initialized with the corresponding data as literal values. This Ruby script does exactly that: | ||
Line 464: | Line 390: | ||
end | end | ||
end | end | ||
=== Metaprogramming in SmallTalk === | === Metaprogramming in SmallTalk === | ||
Smalltalk is an dynamically typed, object-oriented, reflective programming language. As | Smalltalk is an dynamically typed, object-oriented, reflective programming language. As reflection is one feature of smalltalk metaprogramming can be done using this feature. In SmallTalk run time code generation is done by manipulating strings. [http://groups.google.com/group/comp.lang.smalltalk/browse_thread/thread/bb0519b6b4c3e850?pli=1 12] | ||
*'''''Example :''''' | *'''''Example :''''' | ||
Line 481: | Line 405: | ||
Transcript show: newInstance theAnswer; cr. "shows 42" | Transcript show: newInstance theAnswer; cr. "shows 42" | ||
=== Metaprogramming in | === Metaprogramming in C++ === | ||
Metaprogramming in C++ is done by using a procedure called [http://www.idt.mdh.se/kurser/cd5130/msl/2006lp4/reports/drafts/meta_programming.pdf template metaprogramming]. With templates it’s possible to make the compiler pre-evaluate a part of the | |||
program during compile-time, which is called template metaprogramming. It can be viewed as ”programming at compile-time” also. The compiler generates everything from compile-time constants to advanced data structures. Template metaprogramming can create highly optimized programs doing advanced computations at compile-time instead of run-time which leads to faster programs. By moving | |||
computations to compile-time, the program compile-time increases. | |||
These Metaprograms run before the load time of the code they manipulate. Precompilers: C/C++ precompiler. Open compilers: like writing transformations on AST (hygenic macros). Two level languages: C++ templates[http://aszt.inf.elte.hu/~gsd/halado_cpp/ch06s04.html 3]. | These Metaprograms run before the load time of the code they manipulate. Precompilers: C/C++ precompiler. Open compilers: like writing transformations on AST (hygenic macros). Two level languages: C++ templates[http://aszt.inf.elte.hu/~gsd/halado_cpp/ch06s04.html 3]. | ||
Line 530: | Line 451: | ||
// unruh.cpp 30: conversion from enum to D<17> requested in Prime_print | // unruh.cpp 30: conversion from enum to D<17> requested in Prime_print | ||
// unruh.cpp 30: conversion from enum to D<19> requested in Prime_print | // unruh.cpp 30: conversion from enum to D<19> requested in Prime_print | ||
== Comparison of metaprogramming in different languages == | |||
Different languages use different techniques for metaprogramming based on their structure and syntactical support. The most common ways are use of eval function, macro exapansion, reflection for dynamic code generation, use of domain specific languages or template metaprogramming. Lisp is the root of all languages which uses eval. Languages such as Perl, Ruby, Python, PHP, and JavaScript borrowed the verb “eval” from Lisp. The language cUrl also uses its evaluate() function for metaprogramming. Use of macro and DSL is also common in Lisp for metarogramming. It seems out of all metaprogramming languages Lisp is the oldest one and supports strong feature for it. There are other languages like MetaL which is totally designed for metaprogramming. Among all the languages discussed, generally those which have object oriented feature supports metaprogramming mostly. But out of non object oriented languages there are common Lisp, Maude system, Haskell which supports meta programming. But these are research languages and rarely used for commercial purposes. | |||
Another one of the most common metaprogramming scenarios is the creation of classes with attributes and methods that are dynamically generated which is known as metaclass. Python uses this approach for metaprogramming. Languages like ruby, nemerle uses aspect oriented programming also for metaprogramming. | |||
Reflection constitutes another domain of metaprogramming. It’s the ability of a language to inspect its own code – most commonly, to determine what members a given class provides. It can therefore be used to extend the programming language beyond its usual capabilities. Ruby uses reflection for dynamic code generation. Other languages like Maude system, SmallTalk etc. uses the same method of metaprogramming. SmallTalk has [http://vst.ensm-douai.fr/MetaclassTalk Metaclass Talk] a reflective extension of Smalltalk that provides programmers with a [http://www.lisp.org/mop/index.html meta-object protocol (MOP)] to control objects structure (memory allocation and access to instance variables) and behavior (message sends and receptions and method lookup and application). It aims easing experiments of new concepts, programming pradigms and langage extensions. | |||
The languages like C++, D, Haskell etc. uses template metaprogramming. C++ has standard template library included in to the C++ Standard Library which describes containers, iterators, and algorithms. It makes it easier with generic programming and template metaprogramming. | |||
Java does not contain any native constructs for performing metaprogramming as such. But metaprogramming can be accomplished trough the use of a pre-processor, independent of the actual Java language. Some of these are actually implemented as importable java-packages, such as RECODER, where we can create a program and compile it with our regular Java compiler. The output of the program is the new code that we can compile to get the program we were writing. Other, more common approaches in Java are regular pre-processors that parse a source file and create a pure java source file that can be compiled. Some of these, like [http://www.emn.fr/x-info/sudholt/research/metaj/ MetaJ], consist of a mixture of plain Java code that will be copied verbatim | |||
into the output file, as well as special metaprogramming declarations that will control how the output code is created. | |||
Out of different languages like C++, Java and C#, C++ is the only language that has native metaprogramming facilities. Both Java and C# require external pre-processors or complex libraries that are capable of generating code, which then needs to | |||
be compiles separately. The fact is that few programmers know of metaprogramming techniques, make use of them or use them correctly. Nevertheless, metaprogramming can be used to accomplish a lot of good things, and it is a real weakness in a language to exclude such features at a native level. [http://www.idt.mdh.se/kurser/cd5130/msl/2006lp4/reports/drafts/meta_programming.pdf 18] | |||
== Conclusion == | |||
Thus Metaprogramming improves the performance of the code.Instead of being processed by an interpreted script manipulating script-level data structures, the data is processed by the compiled code of the language interpreter manipulating low level data structures directly. This removes the levels of indirection that slow down the execution of scripts, and so results in significant performance improvements. | |||
== Index == | == Index == | ||
Line 543: | Line 484: | ||
* '''[http://en.wikipedia.org/wiki/Type_safety Type-safety] :''' Type safety is a property of some programming languages which indicates the use of a type system to prevent certain erroneous or undesirable program behavior called type errors. | * '''[http://en.wikipedia.org/wiki/Type_safety Type-safety] :''' Type safety is a property of some programming languages which indicates the use of a type system to prevent certain erroneous or undesirable program behavior called type errors. | ||
* '''[http://en.wikipedia.org/wiki/Template_metaprogramming Template metaprogramming] :''' This is a technique in which templates are used by a compiler to generate temporary source code, which is merged by the compiler with the rest of the source code and then compiled. | |||
== References == | == References == | ||
Line 562: | Line 505: | ||
<br>16. http://www.meta-language.net/sample.html#metal -MetaL | <br>16. http://www.meta-language.net/sample.html#metal -MetaL | ||
<br>17. http://en.wikipedia.org/wiki/Maude_system - Reflection based metaprogramming in Maude system | <br>17. http://en.wikipedia.org/wiki/Maude_system - Reflection based metaprogramming in Maude system | ||
<br>18. http://www.idt.mdh.se/kurser/cd5130/msl/2006lp4/reports/drafts/meta_programming.pdf - Metaprogramming in C++ | |||
<br>19. http://www.geeksaresexy.net/2009/07/22/gas-explains-what-is-metaprogramming/ - Metaprogramming in different languages |
Latest revision as of 00:33, 15 October 2009
Metaprogramming
Introduction
The term 'Meta' is just a preposition meaning “after”, “beyond”, or simply “with” – specially it means a higher level of abstraction, especially a self-referential abstraction. That’s the sense in which it is used in the term metaprogramming – modifying programs programmatically, or modifying the programming language itself.
Metaprogramming, is the creation of procedures and programs that automatically construct the definitions of other procedures and programs. Metaprogramming automates some of the tedious and error-prone parts of the programmer's job. It refers to a variety of ways a program has knowledge of itself or can manipulate itself. It's basically "writing code that writes code". To understand metaprogramming, compiler is a very good example. It takes as input, code in some language, and creates code for another language.
The most liberal definition of metaprogramming is:
- Compile code generation or Runtime code generation (or both)
- Aspect-Oriented Thinking or Aspect Oriented Programming
- DRY Thinking i.e 'don't repeat yourself'. So when you find any duplication of code write a metaprogram to apply 'DRY'
- Template metaprogramming
- Self modifying code
- Self code generation
How Meta programming is implemented ?
There are many languages that do metaprogramming. In languages like C#, reflection is a form of metaprogramming since the program can examine information about itself. For example, returning a list of all the properties of an object. In languages like ActionScript, we can evaluate functions at runtime to create new programs such as eval("x" + i).DoSomething() would affect an object called x1 when i is 1 and x2 when i is 2.
Another common form of metaprogramming is when the program can change itself in non-trivial fashions. LISP is well known for this. The program would change another part of the program based on its state. This allows a level of flexibility to make decisions at runtime that is very difficult in most popular languages today. It is also worth noting that back in the good old days of programming in straight assembly, programs that altered themselves at runtime were necessary and very commonplace.
It can be implemented by using any of these and in combination:
- Reflection
- DSLs (Domain Specific Languages)
- (.NET) or Annotations (Java)
- Generics (.NET/Java)
- Templates (C++, Haskell)
- Method_missing (Ruby)
- Closures / first class functions / delegates
- AOP - Aspect Oriented Programming
Different languages that supports Metaprogramming
- Curl
- D
- Groovy
- Haskell
- Lisp
- Maude system
- MetaL
- Nemerle
- Perl
- Python
- Ruby
- Smalltalk
- C++
Metaprogramming in Curl
Curl is an object oriented programming language mainly designed for interactive web content and widely used in enterprise B2B or B2C applications. It is reflection oriented language, so metaprogramming is done in Curl using reflection. Using evaluate() procedure an application can create Curl source code at run time and cause it to be compiled and executed in a chosen environment (package).
- Example:7
evaluate (proc) public {evaluate exp:any, base-url:Url = unspecified-url, package:#OpenPackage = null }:any
In the above example, exp is an expression, which should be a StringInterface or CurlSource containing the source to be evaluated or an url specifying a file from which the source should be read. Package is the environment for evaluation and base-url is the base-url for the code to be evaluated.
Metaprogramming in D
D is an object oriented, multiparadigm programming language mostly influenced from C++. It has redesigned some aspects of C++ and also influenced features from other programming languages like Java and C#. In D, metaprogramming is supported by a combination of templates, tuples, string mixins and compile time function execution. Following are examples of template metaprogramming and metaprogramming using string mixins.
- Example: Template metaprogramming 6
Following example is a template crypt where the encryption adds a constant value 5 to the string. D has built in 'static if' statement used for compile-time conditional construct.
import std.string; import std.stdio; template makechar(int c) { const char [] makechar= x"000102030405060708090a0b0c0d0e0f e0e1e2e3e4e5e6e7e8e9eaebecedeeef f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff"[c..c+1]; } template crypt(char[] n, int val) { static if (n.length == 1) const char[] crypt = makechar!(n[0] + val); else const char[] crypt = makechar!(n[0] + val) ~ crypt!(n[1..n.length], val); } int main() { char[] str = crypt!("This is metaprogramming in D", 5); printf("%.*s\n", str); return 0; }
- Example: String mixin metaprogramming 5
D code is generated using string operations at compile time using string mixins, combined with compile-time function execution. This can be used to parse domain-specific languages to D code, which will be compiled as part of the program.
import FooToD; // hypothetical module which contains a function that parses Foo source code and returns equivalent D code void main() { mixin(fooToD(import("example.foo"))); }
Metaprogramming in Groovy
Groovy is an object oriented programming language for Java platform with dynamic language features. It is also used for scripting. Groovy has extensive metaprogramming facilities and one such addition is the ExpandoMetaClass, which enables us to do a host of copasetic things, including adding methods to normal Java objects (like String). It uses metaprogramming in the following way.
Below example demonstrates using classes from the Jakarta Commons Lang package for metaprogramming. All of the methods in org.apache.commons.lang.StringUtils coincidentally follow the Category pattern — static methods that accept a String as the first parameter. This means that we can use the StringUtils class right out of the box as a Category.1
import org.apache.commons.lang.StringUtils class CommonsTest extends GroovyTestCase{ void testStringUtils(){ def word = "Introduction" word.metaClass.whisper = {-> delegate.toLowerCase() } use(StringUtils, StringHelper){ //from org.apache.commons.lang.StringUtils assertEquals "Intro...", word.abbreviate(8) //from the StringHelper Category assertEquals "INTRODUCTION", word.shout() //from the word.metaClass assertEquals "introduction", word.whisper() } } } class StringHelper{ static String shout(String self){ return self.toUpperCase() } }
Metaprogramming in Haskell
Haskell is a statically typed, general purpose, standardized, purely functional programming language. It is not an object oriented programming language. It supports metaprogramming by compile time code generation using templates. Templates are type-safe compile-time meta-programming. Compile time manipulation of code gives Haskell a lot of power. Template Haskell is used by programmers for conditional compilation which is extremely useful for compiling a single program for different platforms, or with a different configuration, or with different debugging options. Following is an example of template Haskell.
- Example : 14
This example is to select an element from a tuple of arbitrary size. The '$' says 'evaluate at compile time'.
> $(sel 2 3) ('x','y','z') 'y' > $(sel 3 4) ('x','y','z','w') 'z' sel :: Int -> Int -> ExpQ sel i n = [| \x -> $(caseE [| x |] [alt]) |] where alt :: MatchQ alt = match pat (normalB rhs) [] pat :: Pat pat = tupP (map varP as) rhs :: ExpQ rhs = varE(as !! (i -1)) -- !! is 0 based as :: [String] as = ["a" ++ show i | i <- [1..n] ]
Alternately:
sel' i n = lamE [pat] rhs where pat = tupP (map varP as) rhs = varE (as !! (i - 1)) as = [ "a" ++ show j | j <- [1..n] ]
Metaprogramming in Lisp
Lisp is a multiparadigm, functional, procedural and reflective programming language. Lisp provides DSL for metaprogramming. In lisp Metaprogramming is done using a construct called a "macro".
- Example : 10
Let us consider, the following s-expression for a to-do list in Lisp. Suppose we're writing a to-do manager application. We keep our to-do items serialized in a set of files and when the program starts up we want to read them and display them to the user. One way is to read the entire list and show the relevant data to the user. Instead of writing code to walk the s-expression that stores our data we could write a macro that allows us to treat data as code.
(todo "Homeworks" (item (priority high) "Homework for subject 1.") (item (priority medium) "Homework for subject 2.") (item (priority medium) "Homework for subject 3."))
Corresponding macros for the to-do list items that will get called by lisp compiler and will transform the to-do list into code will be as follows. Now the to-do list will be treated as code and will be executed. Suppose all we want to do is print it to standard output for the user to read.
(defmacro item (priority note) '(block (print stdout tab "Priority: " ~(head (tail priority)) endl) (print stdout tab "Note: " ~note endl endl)))
Metaprogramming in Maude System
Maude system is an implementation of rewriting logic and an heavy emphasis on reflection based metaprogramming. The following example illustrates it using a set of rewrite rules using reflection. Here NAT represents the set of natural numbers. s(0) indicates successor of 0 i.e. 1. the rewriting rules r1 are created using the reflection of itself.
- Example: 17
mod PERSON is including NAT . *** NAT is the set of natural numbers sort Person . sort State . op married : -> State [ctor] . op divorced : -> State [ctor] . op single : -> State [ctor] . op engaged : -> State [ctor] . op dead : -> State [ctor] . op person : State Nat -> Person [ctor] . var N : Nat . var S : State . rl [birthday] : person (S, N) => person (S, N + s(0)) . rl [get-engaged] : person (single, N) => person (engaged, N) . rl [get-married] : person (engaged, N) => person (married, N) . rl [get-divorced] : person (married, N) => person (divorced, N) . rl [las-vegas] : person (S, N) => person (married, N) . rl [die] : person (S, N) => person (dead, N) . endm
In the above example a person still is the same person after he is marriedhis marital status has been changed. This is shown in the rewrite rules. These rules do not have to be confluent and terminating so it does matter a great deal what rules are chosen to rewrite the term. The rules are applied at 'random' by the Maude system, indicating no surity that one rule is applied before another rule and so on.
Metaprogramming in MetaL
MetaL is shorthand for Meta-programming Language. MetaL programs source code is based on XML. MetaL compiler engine can be used to generate the same program from MetaL source code to potentially any target language. Currently supported target languages are PHP, Java and Perl.
The MetaL compiler engine traverses XML source code and executes actions associated with each command tag. Actions may be immediate, like creating a file, or generating some output, like the lines of code of a program in a given target language.Command actions are implemented by compiler modules. Such modules implement the different kinds of commands of the language, like execution flow, variable and expression manipulation, object oriented programming classes, etc.. Each module is responsible for generating the code in each of the supported target languages
Source code of the Hello world! program written in MetaL.
<?xml version="1.0"?> <output> <script> <string>Hello World! </string> </script> </output>
Metaprogramming in Nemerle
Nemerle is statically typed, high-level, object-oriented programming language for the .NET platform. It has macro facility, aspect oriented programming features which give Nemerle power of metaprogramming.
- Example: 15
For selecting data from employee table based on firstname as parameter the general way to write code in Nemerle is as follows.
string sql = "SELECT name, address FROM employee WHERE name = :a"; NpgsqlCommand dbcmd = new NpgsqlCommand (sql, dbcon, dbtran); dbcmd.Parameters.Add("a", myparm); NpgsqlReader reader = dbcmd.ExecuteReader(); while(reader.Read()) { string name = reader.GetString (0); string address = reader.GetString (1); System.Console.WriteLine ("Name: {0} {1}", name, address) } reader.Close(); dbcmd.Dispose();
In the above scenario, database connection is tested during run time not compile time and another drawback is the compiler has to remember the variable names during compilation and store them. To avoid this a macro can be used as follows.
ExecuteReaderLoop ( "SELECT name, address FROM employee WHERE name = $myparm", dbcon, { System.Console.WriteLine ("Name: {0} {1}", name, address) });
Metaprogramming in Perl
Below code is for reducing the amount of boiler-plate code necessary for writing Perl subs.Here is an example of a module that uses "handy" new module.13
package SomeTestPackage2; use strict; use warnings; use Data::Dumper; use constant IVARS => qw[$none $href $aref $non_empty_aref]; BEGIN { use base qw[SomeAttributes2]; __PACKAGE__->import( IVARS ); } use constant custom_aref => q{ confess( '!!!package!!! !!!ivar!!! must be an array ref' ) if 'ARRAY' ne ref !!!ivar!!!; }; sub self_sub : Method( qw/ $none :none $href :href $aref custom_aref $non_empty_aref :non_empty_aref / ) { # return ( ref $self, ref $href, ref $aref, scalar @{ $non_empty_aref } ); } 1;
The previous code snip shows how the meta programming interface works for the most part.First,we specify a list of instance variables,that we would like to pull into subs. This pre-declaration is necessary for syntax reasons.Next, we can specify "custom" attributes as package subs if we desire. The difference between a custom attribute and a canned/common one is that canned attributes begin with a colon and custom do not.Now, in our sub attribute we specify a list of variables we would like pulled into our sub along with any attributes we would like called on them.
This is the test script running the prior code
#!/usr/bin/perl use strict; use warnings; use Test::More qw[no_plan]; use SomeTestPackage2; do { # Successfull call to self_sub my( $test1 ) = bless( {}, 'SomeTestPackage2' ); my( @self_sub ) = $test1->self_sub( 'hi', { }, [ ], [ 1 ] ); ok( $self_sub[0] eq 'SomeTestPackage2', 'ref $self' ); ok( $self_sub[1] eq 'HASH', 'ref $href' ); ok( $self_sub[2] eq 'ARRAY', 'ref $aref' ); ok( $self_sub[3] > 0, 'non empty array' ); };
Here is the output of running the previous script.
$ perl test2.pl ok 1 - ref $self ok 2 - ref $href ok 3 - ref $aref ok 4 - non empty array 1..4
Metaprogramming in Python
Python supports metaprogramming through metaclasses, decorators and descriptors.
- Metaclasses considered as "the class of a class" enable the creation of new “types of classes” which can customize the class creation process and/or add class methods and attributes.
- Decorators look like Scala annotations but are applied only to function definitions and yield (possibly transformed) functions.
- Descriptors provide a very concise way to add custom properties to classes that invoke logic but syntactically manipulated as regular attributes. They can also carry extra information (such as titles, database column mappings, etc) that can be useful when accessed through reflection.
class _AutoWriteDescriptor(object): Descriptor for AutoWrite. Implements an attribute that allows an initial setting then calls the setter. def __init__(self, name, ignore_identical=True, prefix='_set_'): self.name = name self.ignore_identical = ignore_identical self.prefix = prefix def __get__(self, obj, objtype=None): self.check_dictionary(obj) return obj._auto_write_dict[self.name] def __set__(self, obj, new_value): self.check_dictionary(obj) if self.name in obj._auto_write_dict: if (self.ignore_identical is False or new_value is not obj._auto_write_dict[self.name]): attr = getattr(obj.__class__, "%s%s" % (self.prefix, self.name), None) obj._auto_write_dict[self.name] = attr(obj, new_value) else: obj._auto_write_dict[self.name] = new_value def __delete__(self, obj): raise AttributeError("cannot delete AutoWrite attribute") def check_dictionary(self, obj): if getattr(obj, "_auto_write_dict", None) is None: raise AttributeError( AutoWrite dictionary missing. Class %s probably does not call AutoWrite.__init__ % obj.__class__)
Above example is a simple Descriptor. The only thing to understand about Descriptors is that they are attributes. So the methods above live on the attribute, not on the main object. When Python accesses an attribute it checks and, if it's a Descriptor, it invokes the appropriate method (get if the attribute is being read, set if its value is being changed, delete if the attribute is being deleted)11.
The advantage of doing things this way is twofold. First, we end up uniting methods and functions.Second,we have an efficient way to change how certain attributes behave. This is much better than changing the main object's __getattr__ and _setattr__ because the extra code is only invoked for the "special" attributes - there no speed penalty otherwise.
If the value is being read, the actual value is pulled from a dictionary that is stored on the main object.If the value is being set for the first time (ie when the object is populated from the database) the value is stored. Subsequent updates (ie when the object is modified via the user interface) call the setter method.
Metaprogramming in Ruby
Ruby is a dynamically-typed, interpreted and object oriented language. Ruby is by default a metaprogramming language as the syntaxes if Ruby support that. Ruby has the following features which makes Ruby a metaprogramming language.
- Dynamic and reflective
- Everything is open to change
- Blocks allow writing new control structures
- Add continuations to really get fancy
- Most declarations are executable statements
Metaprograms in Ruby are written by using eval function, or using macro or using DSL.
- Example:
We have a C program that needs to include a PNG image, but for some reason, the deployment platform can accept one file only, the executable file. Thus, the data that conforms the PNG file data has to be integrated within the program code itself. To achieve this, we can read the PNG file beforehand and generate the C source text for an array declaration, initialized with the corresponding data as literal values. This Ruby script does exactly that:
INPUT_FILE_NAME = 'ljlogo.png' OUTPUT_FILE_NAME = 'ljlogo.h' DATA_VARIABLE_NAME = 'ljlogo' File.open(INPUT_FILE_NAME, 'r') do |input| File.open(OUTPUT_FILE_NAME, 'w') do |output| output.print "unsigned char #{DATA_VARIABLE_NAME}[] = {" data = input.read.unpack('C*') data.length.times do |i| if i % 8 == 0 output.print "\n " end output.print '0x%02X' % data[i] output.print ', ' if i < data.length - 1 end output.puts "\n};" end end
Metaprogramming in SmallTalk
Smalltalk is an dynamically typed, object-oriented, reflective programming language. As reflection is one feature of smalltalk metaprogramming can be done using this feature. In SmallTalk run time code generation is done by manipulating strings. 12
- Example :
In this example, metaprogramming is done by creating anonymous classes and giving them behavior, then instantiate them run time.
| newClass newInstance | newClass := Behavior new. "create anonymous behavior" newClass compile: 'theAnswer ^42'. "Add a method for instances" newInstance := newClass new. "create an instance" Transcript show: newInstance theAnswer; cr. "shows 42"
Metaprogramming in C++
Metaprogramming in C++ is done by using a procedure called template metaprogramming. With templates it’s possible to make the compiler pre-evaluate a part of the program during compile-time, which is called template metaprogramming. It can be viewed as ”programming at compile-time” also. The compiler generates everything from compile-time constants to advanced data structures. Template metaprogramming can create highly optimized programs doing advanced computations at compile-time instead of run-time which leads to faster programs. By moving computations to compile-time, the program compile-time increases.
These Metaprograms run before the load time of the code they manipulate. Precompilers: C/C++ precompiler. Open compilers: like writing transformations on AST (hygenic macros). Two level languages: C++ templates3.
template <int i> struct D { D(void *); operator int(); }; template <int p, int i> struct is_prime { enum { prim = (p%i) && is_prime<(i>2?p:0), i>::prim }; }; template <int i> struct Prime_print { Prime_print<i-1> a; enum { prim = is_prime<i,i-1>::prim }; void f() { D d = prim; } }; struct is_prime<0,0> { enum { prim = 1 }; }; struct is_prime<0,1> { enum { prim = 1 }; }; struct Prime_print<2> { enum { prim = 1 }; void f() { D<2> d = prim; } }; void foo() { Prime_print<10> a; }
// output: // unruh.cpp 30: conversion from enum to D<2> requested in Prime_print // unruh.cpp 30: conversion from enum to D<3> requested in Prime_print // unruh.cpp 30: conversion from enum to D<5> requested in Prime_print // unruh.cpp 30: conversion from enum to D<7> requested in Prime_print // unruh.cpp 30: conversion from enum to D<11> requested in Prime_print // unruh.cpp 30: conversion from enum to D<13> requested in Prime_print // unruh.cpp 30: conversion from enum to D<17> requested in Prime_print // unruh.cpp 30: conversion from enum to D<19> requested in Prime_print
Comparison of metaprogramming in different languages
Different languages use different techniques for metaprogramming based on their structure and syntactical support. The most common ways are use of eval function, macro exapansion, reflection for dynamic code generation, use of domain specific languages or template metaprogramming. Lisp is the root of all languages which uses eval. Languages such as Perl, Ruby, Python, PHP, and JavaScript borrowed the verb “eval” from Lisp. The language cUrl also uses its evaluate() function for metaprogramming. Use of macro and DSL is also common in Lisp for metarogramming. It seems out of all metaprogramming languages Lisp is the oldest one and supports strong feature for it. There are other languages like MetaL which is totally designed for metaprogramming. Among all the languages discussed, generally those which have object oriented feature supports metaprogramming mostly. But out of non object oriented languages there are common Lisp, Maude system, Haskell which supports meta programming. But these are research languages and rarely used for commercial purposes.
Another one of the most common metaprogramming scenarios is the creation of classes with attributes and methods that are dynamically generated which is known as metaclass. Python uses this approach for metaprogramming. Languages like ruby, nemerle uses aspect oriented programming also for metaprogramming.
Reflection constitutes another domain of metaprogramming. It’s the ability of a language to inspect its own code – most commonly, to determine what members a given class provides. It can therefore be used to extend the programming language beyond its usual capabilities. Ruby uses reflection for dynamic code generation. Other languages like Maude system, SmallTalk etc. uses the same method of metaprogramming. SmallTalk has Metaclass Talk a reflective extension of Smalltalk that provides programmers with a meta-object protocol (MOP) to control objects structure (memory allocation and access to instance variables) and behavior (message sends and receptions and method lookup and application). It aims easing experiments of new concepts, programming pradigms and langage extensions.
The languages like C++, D, Haskell etc. uses template metaprogramming. C++ has standard template library included in to the C++ Standard Library which describes containers, iterators, and algorithms. It makes it easier with generic programming and template metaprogramming.
Java does not contain any native constructs for performing metaprogramming as such. But metaprogramming can be accomplished trough the use of a pre-processor, independent of the actual Java language. Some of these are actually implemented as importable java-packages, such as RECODER, where we can create a program and compile it with our regular Java compiler. The output of the program is the new code that we can compile to get the program we were writing. Other, more common approaches in Java are regular pre-processors that parse a source file and create a pure java source file that can be compiled. Some of these, like MetaJ, consist of a mixture of plain Java code that will be copied verbatim into the output file, as well as special metaprogramming declarations that will control how the output code is created.
Out of different languages like C++, Java and C#, C++ is the only language that has native metaprogramming facilities. Both Java and C# require external pre-processors or complex libraries that are capable of generating code, which then needs to be compiles separately. The fact is that few programmers know of metaprogramming techniques, make use of them or use them correctly. Nevertheless, metaprogramming can be used to accomplish a lot of good things, and it is a real weakness in a language to exclude such features at a native level. 18
Conclusion
Thus Metaprogramming improves the performance of the code.Instead of being processed by an interpreted script manipulating script-level data structures, the data is processed by the compiled code of the language interpreter manipulating low level data structures directly. This removes the levels of indirection that slow down the execution of scripts, and so results in significant performance improvements.
Index
- Multiparadigm language : Multiparadigm languages provides a framework in which programmers can work in a variety of styles, freely intermixing constructs from different paradigms. Examples are C, C++, Java, Ruby, Oz, Visual Basic, Perl, Python etc. The design goal of multiparadigm languages is to allow programmers to use the best tool for a job, admitting that a single paradigm can not solve all problems in most efficient or easiest way.
- Template metaprogramming : Template metaprogramming is a metaprogramming technique in which templates are used by a compiler to generate temporary source code, which is merged by the compiler with the rest of the source code and then compiled.
- Reflection oriented language : In reflective programming languages computer programs can observe and modify its own structure and behavior.
- Stack based programming language : This type of languages uses stack based models for passing parameters.
- s-expression : It is called symbolic expression i.e. an way to represent semi-structured data in human-readable textual form mostly made of symbols and lists. It is used in Lisp family of languages.
- Type-safety : Type safety is a property of some programming languages which indicates the use of a type system to prevent certain erroneous or undesirable program behavior called type errors.
- Template metaprogramming : This is a technique in which templates are used by a compiler to generate temporary source code, which is merged by the compiler with the rest of the source code and then compiled.
References
1. http://www.ibm.com/developerworks/java/library/j-pg06239.html
2. http://stackoverflow.com/questions/514644/what-exactly-is-metaprogramming
3. http://aszt.inf.elte.hu/~gsd/halado_cpp/ch06s04.html
4. http://en.wikipedia.org/wiki/Multi-paradigm_programming_language#Multi-paradigm_programming_language - Multi-paradigm_programming_language
5. http://en.wikipedia.org/wiki/D_%28programming_language%29 - D Programming language
6. http://www.the-interweb.com/serendipity/index.php?/archives/66-Template-meta-programming-in-D.html - Template metaprogramming in D
7. http://developers.curl.com/userdocs/CurlDocs.htm#docs/en/api-ref/evaluate.html - Curl evaluation fumction
8. http://www.curl.com/pdf/content-language.pdf - Curl a content language for the Web
9. http://en.wikipedia.org/wiki/Forth_%28programming_language%29 - Meta compilation in Forth
10. http://www.defmacro.org/ramblings/lisp.html - Metaprogramming in Lisp
11. http://www.acooke.org/cute/PythonMeta0.html - Metaprogramming in Python
12. http://groups.google.com/group/comp.lang.smalltalk/browse_thread/thread/bb0519b6b4c3e850?pli=1 - Metaprogramming in SmallTalk
13. http://www.perlmonks.org/?node_id=579458 - Perl Metaprogramming
14. http://www.haskell.org/haskellwiki/Template_Haskell - Metaprogramming in Haskell
15. http://en.wikipedia.org/wiki/Nemerle#New_language_constructs - Metaprogramming using macro in Nemerle
16. http://www.meta-language.net/sample.html#metal -MetaL
17. http://en.wikipedia.org/wiki/Maude_system - Reflection based metaprogramming in Maude system
18. http://www.idt.mdh.se/kurser/cd5130/msl/2006lp4/reports/drafts/meta_programming.pdf - Metaprogramming in C++
19. http://www.geeksaresexy.net/2009/07/22/gas-explains-what-is-metaprogramming/ - Metaprogramming in different languages