CSC/ECE 517 Fall 2010/ch1 n 00

Closures in Ruby vs. closures in other languages - Introduction to Closure

Closure is defined in number of different ways by different people. Below shows some of the most commonly used definition.

Wikipedia defines Closure as follows:

A Closure is a first-class function with free variables that are bound in the lexical environment.

An easy to understand definition is from 5:

• It can be passed around as a value and
• executed on demand by anyone who has that value, at which time
• it can refer to variables from the context in which it was created (i.e. it is closed with respect to variable access, in the mathematical sense of the word "closed").

You will find lot of discussions and debates about what Closure should be and how it should work. For more details and historic development of Closure is available here 1

In simple words, you can imagine Closure as a special function pointer, which not only points to a block of executable code, but can also point to the variables that are present in the context in which closure was created.

A Simple Example using Scheme

(define (add a)
 (lambda (b)
   (+ a b)))

(define add6 (add 6))
(add6 4) returns 10

When you call add with parameter a i.e 6 here that value is contained in the closure of the returned function defined using add6. Closure is just a mirror of the stack stored before creating the lambda function and when the function is re-executed the stack is restored to the state before executing the function. So the value 6 is still available when the lambda function executes.

What Closure is not

This section attempts to clarify some of the general misconceptions about Closure. Those who have never experienced Closure, might relate things close to Closure as Closure.

• C Programmer: Might tend to think Closures are like Function Pointers. But that is not true, Closure is more than function pointers, because function pointers does not close local variables in the context. Function pointers are not considered First-Class Objects.
• Java Programmer: Inner Classes are similar to Closures but not Closures exactly. Inner class can access only instance variables of the surrounding class and has only READ-ONLY access to the local variables. The local variables must be marked final to make them available for the inner class to use them.

Why Closure is needed?

If you think, Closure is not a mandatory requirement for a language. Many popular programming languages like C, Java, C++ does not have Closure. Closure adds more convenience to a language, at the same time it makes it difficult for first time users. With Closure, you can really write short code and avoid passing too many arguments, since Closure closes the local variables.

When to use Closure?

Closure is a very handy tool which makes the code less verbose. You can use Closure where you used Function Pointers, Anonymous Functions(), Anonymous Inner Classes. This is a very common piece of code in Java for creating a Thread.

Thread t = new Thread(new Runnable() {
  public void run() {
  // thread logic
  }
});

The above piece of code if re-written with Closures take a very simple form as follows:

Thread t = new Thread(#() {
   // thread logic
});

Above statement is written using one of proposed Closure syntax addition for Java 7 (Java does not have Closures yet). You can see that the code is lot less verbose and neat. When Closures are available in Java, you can see that you can do away with most of the Callback classes used in Java.

Closures in the form of Delegates are very actively used in C#. Delegates are used to add and remove event listener in C# elegantly. To read more about the use of Delegates in C# see 3.

Closure vs Lambda

Closure and Lambda expressions are very similar concepts. Both represents a block of executable code, but lambda expression does not have to close the context. Lambda expression can be viewed as unnamed anonymous function. Unlike Closure, Lambda expressions are very loosely defined in Computer Science.

Closure too represents a block of executable code, but Closure closes the context and makes available the variables that are used inside the closure, even after the life time of those stack variables.

We will see later that there is a very subtle difference between Closure and Lambda in Ruby, though both Closure and Lambda encloses the context.

Closure in Ruby

A closure in Ruby as mentioned by Yukihiro Matsumoto4, the creator of Ruby "A closure is a nameless function the way it is done in Lisp. A closure object has the code to run, the executable, state around the code and the scope.So you capture the environment, namely the local variables, in the closure. As a result, you can refer to the local variables inside a closure. Even after the function has returned, and its local scope has been destroyed, the local variables remain in existence as part of the closure object. When no one refers to the closure anymore, it's garbage collected, and the local variables go away."

Since we spent considerable amount of time in class discussing about Ruby, We will spend more time on discussing closure in Ruby too.

Probably when you are reading this, you might not be very clear about what Closure, Blocks and Proc. Are they different? How do they work? This section tries to answer those questions. If, we have done a good job, probably you should understand what they are after reading this section.

Block is a unnamed piece of code, which can be called. Blocks is in fact a closure since it has access to the local variables in the context where it was defined.

Closure is created in Ruby as Proc blocks:

def createIncrementer(step) # 1
  return Proc.new { |x| # 2
                    return x + step # 3
         }
end # 4

by5 = createIncrementer(5) # 5
puts by5.call(15) # prints 20 # 6 

Let us go through this code line by line and see what happens as Ruby interpreter executes the code.

Line 1 - Define a function which is going to return a closure.

Line 2 - We are returning the Closure. Closures are represented by Proc block in Ruby. We are creating a new Proc object which takes a block of executable code as argument. |x| means that the block of code, takes a single argument. The block of code returns the value x incremented by step. It is worth observing that closure refers to the local variable "step" in the code block.

Line 5 - use create incrementer function to create a Closure.

Line 6 - As explained above Closures are represented by Proc object. We need to use call method of the Proc object to execute the code. Whatever value we pass to the call function will be passed on the Closure block as variable x (line 2)

There is a difference in the scope of the variables with same name. The below example illustrates this.

def thrice_x
   x = 50
   yield
   yield
   yield
   puts "x inside thrice_x: #{x}"
end

def test_closure
  x = 5
  thrice_x { x += 1 }
  puts "value of outer x after: #{x}"
end

test_closure()

Output is: x inside thrice_x: 50 value of outer x after: 8

From the above output we can understand that x in thrice_x { x += 1 } refers to the x that is defined inside test_closure function, ie., the context where Block is defined and not from where it is executed.

Just as a side note, block can also be defined using do..end instead of { .. } it means the same.

Consider another example below for Closure. We all know about the synchronized block in Java, but with Closure in Ruby, we have lot more freedom of designing our own blocks.

def lock_and_run
  # try locking
  success = false
  yield(success)
  # unlock
end

lock_and_run { |x|
  if x==true
    # do operation
    puts "Doing dangerous operation!"
  else
    puts "Unable to lock resource"
  end
}

The block of code, enclosed in lock_and_run function, first attemps to lock an resource, if it were able to lock it successfully (reflected by success flag), it calls the block of code with the success flag. Change success to true and false to see what we are talking about.

Difference between Ruby Block and Ruby Proc

Ruby philosophy is everything in Ruby is an object, but not a Block of code. Both Block and Proc represents a block of executable code, but Proc is presented by a class and hence a block of proc is an object and hence can be passed around as arguments, whereas Blocks are not objects, so they cannot be based as argument. [6]

dash_printer = Proc.new { puts "-------------" };

dash_printer.call

The above is a perfectly valid Proc, but below code is not valid. Since block is not an object, you don’t have any functions like call to execute the block of code.

dash_printer = { puts "-------------" };

dash_printer.[ no function to call?? ]

Procs and Lambdas

As defined in [6], the relationship between procs, blocks and lambdas can be described as below: “Blocks are syntactic structures in Ruby; they are not objects, and cannot be manipulated as objects. It is possible, however, to create an object that represents a block. Depending on how the object is created, it is called a proc or a lambda. Procs have block-like behavior and lambdas have method-like behavior. Both, however, are instances of class Proc.” - David Flanagan; Yukihiro Matsumoto

Another way of creating a proc block is as follows. Taken from [6]:

1. This method creates a proc from a block

def makeproc(&p) # block can be received as argument using the & symbol.
  p
end

We can create the proc object as follows:

adder = makeproc {|x,y| x+y }

Lambda:

Lambda expressions are very similar to Procs. There are created using the following syntax;

doubler = lambda {|x| x+x}

You can call doubler using doubler.call(10) returning 20

The above can also be created using the below syntax from Ruby 1.9 [6]:

doubler = ->(x) { x+x }

Arity of proc

Arity of a proc or lambda is defined as the number of arguments expected by the proc or lambda.

lambda { | | }.arity --- is 0

lambda { | a, b | }.arity --- is 2

Difference between Proc and Lambda

Another important difference between lambda and proc is the way return works and as a Ruby programmer, we all must be aware of the difference.

Lambda block is evaluated as though its call to the function. Very similar to the way function call works. Proc block is evaluated as though the function is inlined. We can better understand the above with the below example. When a return statement is executed inside a Proc block, it actually returns from the function which called the proc block.

def level2Builder(message) lambda { puts message; return } end def level1 puts "level1" p = level2Builder("call level2") p.call puts "back to level1" end level1	def level2Builder(message) proc { puts message; return } end def level1 puts "level1" p = level2Builder("call level2") p.call puts "back to level1" end level1
Output level1 call level2 back to level1	Output level1 call level2 understanding_blocks.rb:26:in `block in level2Builder': unexpected return (LocalJumpError)

Perl

Perl has extensive support for Closure with a syntax very similar to the Proc block of Ruby.

sub create_incrementer {
       my $step = shift;
       my $inc = sub {
               my $num = shift; 
               return $num + $step; // closes local variable step
       };
       return $inc;
}
my $by5 = create_incrementer(5);
print "10 Increment by 5 is ";
print $by5->(10);

You can see that create_incrementer creates a closure which closes the local variable step.The output of the above program is 10 Increment by 5 is 15

Javascript

Javascript supports Closure too. Below is an example of how closure can be created in Javascript

function createIncrementer(step)
{
    return function(val) {
        return val + step;
    }
}
by5 = createIncrementer(5);
by5(10); // returns 15

We must note that Closure does not just closes the value of the variable in the context, but the variable reference itself. The above code, returns an code block, which closes the local variable step.

It is important to understand that Closure can sometime hide Circular Dependencies which can cause memory leaks in Javascript. 7 discusses lot more about memory leaks in Javascript.

C#

C# supports full Closure and Functional Programming starting from version 3.0. Earlier versions of C# had something similar to Closure called Delegates. Annonymous delegates introduced in version 3.0 supports Closure.

namespace closure_csharp
{
   class Program
   {
       delegate int incrementer(int i);
       static incrementer createIncrementer(int step)
       {
           incrementer incr = i => (i + step); // closes local variable step
           return incr;
       }
       static void Main(string[] args)
       {
           incrementer by5 = createIncrementer(5);
           Console.WriteLine(by5(10));
       }
   }
}

We can see the createIncrementer creates a delegates and closes the local variable step.

Python

Python supports Closure as well as Lambda expressions. Below example shows how Closure is created in Python.

def createIncrementer(step):
  def incr(x):
     return step + x # x is closed by incr
  return incr

step5 = createIncrementer(5)
puts step5(20) # returns 25

The above code will print 25. The inner function incr doesn't need to provided with the value step. The inner function can reference the variables of the outer function and hence it will close the local value step.

The disadvantage is the inner functions needs to defined inside the outer function and closures does not work if we simply pass a function object.

Python supports Lambda expressions too. Lambda expression is created using the lambda keyword. This keyword is used to create anonymous functions(unnamed functions).This could be used wherever function objects are required.

Syntax: lambda [lambda expression name]: [expression]

An Example:

squarer = lambda x: x*x
squarer(10) # returns 100

The disadvantage is they are syntactically restricted to a single expression.

As a side note functions in python contains an attribute called func_closure.This contains either None (i.e similar to null) or tuple of cells(i.e similar to list but cannot be modified) that contains bindings for the function's free variables.

Scala

Let us see how closure works in scala using an example.Before that List.map creates a new list but after applying the function.

scala> List(1,2,3,4,5).map{i => i * 2} returns a new list (2,4,6,8,10).

Let see how it is used with closure example.

var incrementor = 10
val adder = (i:Int) => i + incrementor
val mylist1 = List(1,2,3,4,5) map adder
println(mylist1)
val incrementor = 20
val mylist2 = List(1,2,3,4,5) map adder
println(mylist2)

This prints 11,12,13,14,15 and 21,22,23,24,25 respectively.

Here we have incrementor value and then we create an anonymous function and assigned it to variable called adder.Then we use map to apply the adder to each of the value in the list.After changing the incrementor value we apply the map again to the list.

When comes to i, this variable is used with a new value each time whenever adder is used.But incrementor is a refernence to the variable in the enclosing scope.This would create a closure which closes over adder and the external context of variables adder references.Hence whenever incrementor is changed the adder behavior is also changed.

Differences in Closure

As highlighted before, different languages use different construct and different rules for Closure. Some of most commonly seen differences are highlighted below:

• In some languages Closure closes the reference to the variable, where as in some languages Closure just closes the variable value.

For example: Python and Javascript closures closes the reference to the variables, where as Perl closes the variable value. The following example illustrates this with more detail: Python:

squarers = []
for i in xrange(3):
    def func(x): return i*i
    squarers.append(func)

for squarer in squarers :
    print squarer()

Output is: 4 4 4

Perl:

my @squarers = ();

foreach my $i (0 .. 2)
{
    push(@squarers, sub {$i * $i});
}

foreach my $squarer (@squarers)
{
    print $squarer->();
}
     

Output is: 0 1 4

• Different languages differ in the way return statements are executed. Some languages consider Closure call as a function call and return statement will return to the place where Closure is called, but in some other languagues Closure call is similar to inline function call and return statement will return from the outer function which called the Closure.

Conclusion

We explored little bit about what Closure is and how to do it in different programming languages. We discussed Closure in Ruby to a greater extent. Closure is a nice feature which makes the code more readable and less verbose.

External links

• (1) Definitions of Closure
• (2) Definitions of Closure
• (3) Closure in C#
• (4) Interview with Matz
• (5) Closures in Ruby
• The Ruby Programming Language, David Flanagan; Yukihiro Matsumoto, O’Reilly, Chapter 6.5 (6)
• (7) Closure in Javascript
• (8) Programming Scala, Dean Wampler;Alex Payne