CSC/ECE 517 Summer 2008/wiki1 2 itr
Introduction
One of the beloved features of Ruby is the block based Iterator. A Ruby Iterator is simply a method that loops over the contents of an object without exposing its underlying representation. The verb 'iterate' means "do the same thing many times' so 'iterator' means "one which does the same thing many times". It can also be considered as an object that behaves like a generic pointer. The iterator usually references to one particular element in the object collection and then modifies itself so that it points to the next element. Generators are a similar feature in Python. The name came as it is the entity which generate iterators. It allows you to write a function that can return a result and pause, resuming in the same place the next time you call the function. The generator feature in Ruby can be implemented by adding a library class called Generator or external iterator. In python the generator is a feature and part of the language.
Problem Definition
Ruby, like Java, has iterators to facilitate doing operations on each member of a set. Python has generators as well. Describe how generators differ from iterators, and find examples of code sequences using generators that would be awkward with iterators.
Iterator
The word "iterator" means different things in different contexts and programming languages, but it's always got something to do with visiting each one of a set of objects, where "object" doesn't necessarily mean "class instance": Just take "object" to mean "some instance of some data type, somewhere". Iterators may provide additional features and may behave in a different way depending on the languages.
Implementing Iterator
Most of the OOP languages provide ways to make iterations easy, for example some languages provide class for controlling iteration, etc. But Ruby allows the definition of control structures directly. In terms of Ruby, such user-defined control structures are called iterators.
Iterator in Ruby
Iterator in Ruby is simply a method that invokes a block of code. The power is in the code block between the do and end keywords or {...}. Meaning, we can put as much, or little, code in there as needed. And each item being iterated over is passed into the block as a parameter between the pipes. Examples of different iterators are given below.
Using Each [1]
a = [ 1, 2, 3 ]
a.each { |x| print x }
==>123
#The internal implementation of the Array.each method could be defined internally like this:
# def each
# for i in 0...size
# yield(self[i])
# end
# end
x is the local variable in which each value of a is stored. And, each is probably the simplest iterator which yield successive elements of its collection.The above two line code is translated into C.
#include<stdio.h>
main()
{
int s[3]={1,2,3};
int i=0;
while(i<3)
{
printf("%d",s[i]);
i++;
}
}
==>123
The advantage of the block-passing style is that generators are a lot more built-in to the language. For example, the below two codes are equivalent and in neither of those cases is an array instantiated with all members of the range (X..Y):
1.upto(100000).each {|x| puts x}
for i in 1..100000 puts i end
Using Find [2]
a = [ 10,20,30 ]
a.find { |n| n % 5 == 0 }
==>10
The find iterator method in ruby will compare each element using some comparison operator (<, >, ==, etc.) and based on the boolean result (true or false), it will return the first matching value.
Using Collect [3]
a = [ 1, 2, 3, 4, 5 ]
b = a.collect { |n| n + 1 }
==>[2, 3, 4, 5, 6]
Another common iterator is the collect that returns an array of elements that is taken from the corresponding collections. Iterator can return derived values and not only limited to accessing the data stored in arrays and hashes.
Iterator in Python
Python also supports iteration over containers. It is implicitly used in the for statement, in list comprehensions, and in generator expressions. An iteration protocol is defined so that iteration is possible over different objects. Most of the container objects can be looped over using for statement. Example of a typical implicit iteration over a sequence is given below [4].
for element in [1, 2, 3]: print element
The actual internal implementation is that the for statement calls iter() on the container object. The function returns an iterator object that defines the method next(). The method next() then returns the next item one at a time. When there are no more elements, next() raises a StopIteration exception which tells the for loop to terminate. Iterators can also be defined explicitly. An example using explicit iterators is given below [5]
it = iter(sequence)
while True:
try:
value = it.next()
except StopIteration:
break
print value
Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries, and other more specialized forms. The iterator object just have to implement __iter()__ and next(). Ruby iterator has an advantage over Python as it supports code blocks as objects where as Python can use only the for loop for iteration.
Uses
- The main use of iterator is it hides the internal details from the user when manipulating with the objects.
- Iterator is trivially more cleaner and elegant which makes it more easy to maintain compared to the accessing of elements based on indexing.
- Iterators follow a consistent way of iterating through all kinds of data structures, as a result it becomes more readable and reusable.
- Using Iterators, inserting of a new element into the container object is easy even after the iterator has advanced beyond the first element. On the other hand, it is more difficult when using indexing as it requires changing of index numbers.
Generators
A generator looks like a function but behaves like an iterator. Both Ruby and Python support generators. As mentioned in the introduction, Ruby has a Generator library and Python has generator as a fundamental part of the language. As being a language feature it is more robust and syntactically more concise and cleaner. On the other hand, using a library class helps to learn a language more easily and more elegant with the less number of features.
Generators in Python
Generators are a simple and powerful tool for creating iterators. They are written like regular functions and called only once. It then returns an iterator which is the actual iterator with _iter()_ and next() methods. Generator doesn't have to worry about the iterator protocol (__iter()__,.next()..), it just works. Yield statement is used whenever they want to return data. Each time next() is called, the generator resumes where it left-off (it remembers all the data values and which statement was last executed). The generator function does NOT run to completion when it's first called - instead, it only runs until it has a value available to return, at which point it yields that value back and suspends operation until called again to resume. This is described with an example below [6].
>>> def generator1():
... yield "first"
... yield "second"
... yield "third"
>>> gen = generator1()
>>> gen
>>> #No output was produced.
>>> gen.next()
'first' #Function starts executing here.
>>> gen.next()
'second'
>>> gen.next()
'third'
>>> gen.next()
Traceback (most recent call last):
File "", line 1, in ?
StopIteration
First, a generator called generator1 is defined which will yield three values, the strings "first", "second" and "third". When we create a new generator object (gen) it begins the function. Each time you call the generator's next method, it continues the function until the next yield. When the generator reaches the end of the block, or reaches a return statement, it throws a StopIteration exception. A generator is a one time operation. So, the generated data is iterated only once but one can call the generated function again if needed. In this way, Lazy evaluation can be achieved which increases the performance by eliminating unnecessary calculation of values that are never used. Generator can be more powerful as Python has now included the ability to return the data the generator. For example :
x= yield y
From the perspective of the caller of the generator, this statement returns control back to the caller, just as before. From the perspective of the generator, when the execution comes back into the generator, a value will come with it. In this case, the generator saves it into the variable x. The value comes when the caller calls a function called send() to pass a value back into the generator. The function send() behaves just like the function next(), except that it passes a value. This is explained with an example below.
class MyStuff: def __init__(self): self.a = [2,4,6,8,10,12] def iterate(self): i = 0 while i < len(self.a): val = yield self.a[i] if val == None: i += 1 else: if val < 0: val = 0 if val >= len(self.a): val = len(self.a) -1 i = val
Here, send() is used to let the caller of the generator reposition the index in the array. Value received is checked whether it is within the bounds of the array, else the closest boundary is set. The value generator received is tested and stored in 'val'.If val is None, that means the generator received no value (the calling code used next(), or perhaps send(None)).
Generators in Ruby
Some of the reason's why Generator library comes into play where external iterators are implemented are mentioned below.
- The internal iterator in Ruby would fail if the iterator is required to pass through a method that needs access on each of the values returned by that iterator.
- Iterators cannot iterate over more than one collection at a time, they cannot be paused or stopped in the middle.
- They can only implement a single traversal strategy.
Example with the library to iterate over a block is given below [7].
require 'generator'
gen = Generator.new do |result|
result.yield " Start"
3.times { |i| result.yield i}
result.yield "done"
end
while gen.next?
print gen.next,"--"
end
==> Start--0--1--2--done--
An example to print the successive elements from 10 to 20 in ruby and python
Using generator in Python
def countfrom(n):
while True:
yield n
n += 1
for i in countfrom(10):
if i <= 20:
print i
else:
break
==>10,11,12,13,14,15,16,17,18,19,20
Using iterators in Ruby
(10..20).each { |i| print i,"," }
==>10,11,12,13,14,15,16,17,18,19,20
This above example shows how iterators in Ruby are not very elegant.
Using generator in Ruby
require 'generator' gen = Generator.new(10..20) while gen.next? print gen.next, ", " end ==>10,11,12,13,14,15,16,17,18,19,20
With Ruby generators the code flows similar to Python.
Comparisons of Ruby generators and Python generators
Python was intended to be a highly readable language.Ruby can be a highly readable language with the use of external iterators. Ruby's internal iterators aren't always the best solution. Adding the generator or external iterators function helps get over the difficulty of writing the code and helps with the readability. This sort of makes Ruby like a highly readable language like Python. Ruby's code however runs slower than many compiled languages and other major scripting languages such as Python.
examples of code sequences using generators that would be awkward with iterators
Unlike the use of range in Python, Ruby has a data type called Range as well; an object of this type constitutes a set of everything between the start and end of the range (including or not including the limits, depending on additional conditions). Since Range is a subclass of Enumerable, one would intuitively expect that iterating a valid Range object will give you every single object in that set, from start to end. This expectation turns out to be incorrect in Ruby but not in Python as shown in the example:[8]
irb(main):001:0> (1..3).each { |i| puts i }
1
2
3
=> 1..3
irb(main):002:0> (3..1).each { |i| puts i }
=> 3..1 #It fails here.
irb(main):003:0>
In our previous example going from 20 to 10 in Ruby may be difficult.
require 'generator' gen = Generator.new(20..10) while gen.next? print gen.next, ", " end ==>nil