CSC/ECE 517 Spring 2014/ch1 1w1h jg: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
== '''Ruby libraries to load objects of other languages at run time''' ==
== '''Ruby libraries to load objects of other languages at run time''' ==
[http://en.wikipedia.org/wiki/Ruby_(programming_language) Ruby] is a dynamic, reflective, object-oriented, general-purpose programming language. It is easy to extend Ruby with new features by writing code in Ruby. But every now and then extending ruby with low-level languages, such as[http://en.wikipedia.org/wiki/C_(programming_language) C]/[http://en.wikipedia.org/wiki/C%2B%2B C++]/[http://en.wikipedia.org/wiki/Java_(programming_language) Java] is also necessary.


Currently, various kinds of languages codes could be invoked from within ruby. The extension for C/C++ and Java are focused here.
This is an advanced topic. Libraries are being developed where Ruby gems can load libraries (and even native code) of other languages.A few open-source projects like FFI are currently working on this.
 


__TOC__
__TOC__
==Introduction==
===Runtime Library===
In computer programming, a runtime library is the API used by a compiler to invoke some of the behaviors of a runtime system. The runtime system implements the execution model and other fundamental behaviors of a programming language. The compiler inserts calls to the runtime library into the executable binary. During execution (run time) of that computer program, execution of those calls to the runtime library cause communication between the application and the runtime system. This often includes functions for input and output, or for memory management.
The runtime library may implement a portion of the runtime system's behavior, but if one reads the code of the calls available, they typically are thin wrappers that simply package information and send it to the runtime system. However, sometimes the term runtime library is meant to include the code of the runtime system itself, even though much of that code cannot be directly reached via a library call.


== '''Ruby C/C++ extensions<ref>http://java.ociweb.com/mark/NFJS/RubyCExtensions.pdf</ref>''' ==
For example, some language features that can be performed only (or are more efficient or accurate) at runtime are implemented in the runtime system and may be invoked via the runtime library API, e.g. some logic errors, array bounds checking, dynamic type checking, exception handling and possibly debugging functionality. For this reason, some programming bugs are not discovered until the program is tested in a "live" environment with real data, despite sophisticated compile-time checking and pre-release testing. In this case, the end user may encounter a runtime error message.
By extending Ruby with C. The C libraries could be used directly in Ruby applications.
Ruby could call C codes in three ways: interpreter API, RubyInline, SWIG.
=== interpreter API ===
Ruby interpreter is implemented in C, its API can be used and no special API added for interacting with C like Java’s JNI is needed.


The files needed include:
The concept of a runtime library should not be confused with an ordinary program library like that created by an application programmer or delivered by a third party, nor with a dynamic library, meaning a program library linked at run time.
*'''extconf.rb'''
Used to generate Makefile


Platform-specific Makefiles for compiling C extensions to Ruby is needed to be generated firstly.
===libffi===
FFI stands for Foreign Function Interface. A foreign function interface is the popular name for the interface that allows code written in one language to call code written in another language. The libffi library really only provides the lowest, machine dependent layer of a fully featured foreign function interface. A layer must exist above libffi that handles type conversions for values passed between the two languages.<ref>http://sourceware.org/libffi/</ref>
libffi is a foreign function interface library. It provides a C programming language interface for calling natively compiled functions given information about the target function at run time instead of compile time. It also implements the opposite functionality: libffi can produce a pointer to a function that can accept and decode any combination of arguments defined at run time.
libffi is most often used as a bridging technology between compiled and interpreted language implementations. libffi may also be used to implement plug-ins, where the plug-in's function signatures are not known at the time of creating the host application.
Notable users include Python, Haskell, Dalvik, F-Script, PyPy, PyObjC, RubyCocoa, JRuby, Rubinius, MacRuby, gcj, GNU Smalltalk, IcedTea, Cycript, Pawn, Squeak, Java Native Access, Racket,<ref>http://en.wikipedia.org/wiki/Libffi#cite_note-2</ref> Embeddable Common Lisp and Mozilla.<ref>http://en.wikipedia.org/wiki/Libffi#cite_note-3</ref>
On Mac OS X libffi is commonly used with BridgeSupport, which provides programming language neutral descriptions of framework interfaces, and Nu which binds direct Objective-C access from Lisp.
libffi has been widely ported and is released under a MIT license.


A simple sample is like below:
==How does it work?<ref>http://www.rubyinside.com/ruby-ffi-library-calling-external-libraries-now-easier-1293.html</ref>==
"One of the largest problems plaguing Ruby implementations [..] is the ever-painful story of "extensions". In general, these take the form of a dynamic library, usually written in C, that plugs into and calls Ruby's native API as exposed through ruby.h and libruby."


create a file containing the following,
The many compiled bridges between external libraries and Ruby pose a problem for alternate implementations like JRuby, because of the complexity involved in exposing internals of the implementation or expensive serialization in both directions. Instead, an interface is necessary so that instead of developing libraries that act only as unique bridges to others, we can just have one library that provides the interface to any arbitrary library of our choice.
named extconf.rb by convention
 
Ruby already has a library called "dl" that makes it possible to dynamically link external libraries with Ruby. It's a bit arcane though, and Charles points out that it's not widely used "because of real or perceived bugs." Given this, and given the need for an implementation that can be compatible with JRuby, Wayne Meissner has developed "FFI", a new Ruby library that provides "Foreign Function Interface" features to Ruby.
 
A basic demonstration of a Ruby script that uses C's getpid function should be enough to demonstrate the simplicity of FFI:
<pre>
<pre>
require 'mkmf'
require 'ffi'
extension_name = 'name'
 
dir_config(extension_name)
module GetPid
create_makefile(extension_name)
  extend FFI::Library
</pre>
  attach_function :getpid, [], :uint
And use by running
end
<pre>
 
ruby extconf.rb
puts GetPid.getpid
make
</pre>
Then generates
<pre>
.so under UNIX/Linux
.so under Windows when building with Cygwin
.bundle under Mac OS X
</pre>
</pre>


*'''C File'''
===Progress of FFI<ref>http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html</ref>===
C codes to be invoked from Ruby
One of the largest problems plaguing Ruby implementations is the ever-painful story of "extensions". In general, these take the form of a dynamic library, usually written in C, that plugs into and calls Ruby's native API as exposed through ruby.h and libruby. Ignoring for the moment the fact that this API exposes way more of Ruby's internals than it should, extensions present a very difficult problem for other implementations:
 
Do we support them or not?
 
In many cases, this question is answered for us; most extensions require access to object internals we can't expose, or can't expose without extremely expensive copying back and forth. But there's also a silver lining: the vast majority of C-based extensions exist solely to wrap another library.
 
Isn't it obvious what's needed here?
 
This problem has been tackled by a number of libraries on a number of platforms. On the JVM, there's Java Native Access (JNA). On Python, there's ctypes. And even on Ruby, there's the "dl" stdlib, wrapping libdl for programmatic access to dynamic libraries. But dl is not widely used, because of real or perceived bugs and a rather arcane API. Something better is needed.
 
Wayne is one of the primary maintainers of JNA, and has recently spent time on a new higher-performance version of it called JFFI. Wayne also became a JRuby committer this spring, and perhaps his most impressive contribution to date is a full FFI library for JRuby, based on JNA (eventually JFFI, once we migrate fully) and implementing the full set of what we and Evan agreed would be "FFI API 1.0". We shipped the completed FFI support in JRuby 1.1.4.


An example Hello.c is shown below
<pre>
#include <ruby.h>
#include <stdio.h>
// These C functions will be associated with
// methods of a Ruby class on the next page.
static VALUE hello(VALUE self, VALUE arg) {
char* name = RSTRING(arg)->ptr;
printf("Hello %s!\n", name);
return Qnil;
}
static VALUE goodbye(VALUE class) {
printf("Later dude!\n");
return Qnil;
}
// This is called when the Ruby interpreter loads this C extension.
// The part after "Init_" is the name of the C extension specified
// in extconf.rb, not the name of the C source file.
void Init_hello() {
// Create a Ruby module.
VALUE myModule = rb_define_module("MyModule");
// Create a Ruby class in this module.
// rb_cObject is defined in ruby.h
VALUE myClass =
rb_define_class_under(myModule, "MyClass", rb_cObject);
// Add an instance method to the Ruby class.
int arg_count = 1;
rb_define_method(myClass, "hello", hello, arg_count);
// Add a class method to the Ruby class.
arg_count = 0;
rb_define_module_function(myClass, "goodbye", goodbye, arg_count);
}
</pre>
*'''Ruby File'''
Ruby code that invokes C code


An example client.rb is shown as below
The "Passwd" and "Group" structures for functions like 'getpwuid':
<pre>
<pre>
require 'hello'
module Etc
include MyModule # so MyClass doesn't need MyModule:: prefix
  class Passwd < FFI::Struct
obj = MyClass.new # MyClass is defined in C
    layout :pw_name, :string, 0,
obj.hello('Mark') # calling an object method
          :pw_passwd, :string, 4,
MyClass.goodbye # calling a class method
          :pw_uid, :uint, 8,
          :pw_gid, :uint, 12,
          :pw_dir, :string, 20,
          :pw_shell, :string, 24
  end
  class Group < FFI::Struct
    layout :gr_name, :string, 0,
          :gr_gid, :uint, 8
  end
end
</pre>
</pre>


Finally we could build and run
In JRuby 1.1.5, we've taken another step forward with the API, adding support for callbacks. How would we represent a callback you pass into a C function from Ruby?
<pre>
ruby extconf.rb
make
ruby client.rb
</pre>


Then the output will display as
Binding and calling "qsort" with an array of integers:
<pre>
<pre>
Hello Mark!
require 'ffi'
Later dude!
</pre>


=== RubyInline ===
module LibC
[http://www.zenspider.com/ZSS/Products/RubyInline/ RubyInline] allows C code to be imbedded in Ruby code. It automatically determines if the code in question has changed and builds it only when necessary. The extensions are then automatically loaded into the class/module that defines it.
  extend FFI::Library
  callback :qsort_cmp, [ :pointer, :pointer ], :int
  attach_function :qsort, [ :pointer, :int, :int, :qsort_cmp ], :int
end


To setup RubyInline, just use
p = MemoryPointer.new(:int, 2)
<pre>
p.put_array_of_int32(0, [ 2, 1 ])
gem install rubyinline
puts "Before qsort #{p.get_array_of_int32(0, 2).join(', ')}"
LibC.qsort(p, 2, 4) do |p1, p2|
  i1 = p1.get_int32(0)
  i2 = p2.get_int32(0)
  i1 < i2 ? -1 : i1 > i2 ? 1 : 0
end
puts "After qsort #{p.get_array_of_int32(0, 2).join(', ')}"
</pre>
</pre>


=== SWIG ===
What good is having such a library if it doesn't run everywhere? Up until recently, only Rubinius and JRuby supported FFI, which made our case for cross-implementation use pretty weak. Even though we were getting good use out of FFI, there was no motivation for anyone to use it in general, since the standard Ruby implementation had no support.
[http://en.wikipedia.org/wiki/SWIG SWIG] is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages. SWIG is used with different types of target languages including common scripting languages such as Perl, PHP, Python, Tcl and Ruby.  
 
That is, until Wayne pulled another rabbit out of his hat and implemented FFI for C Ruby as well. The JRuby team is proud to announce a wholly non-JRuby library: FFI is now available on Ruby 1.9 and Ruby 1.8.6/7, in addition to JRuby 1.1.4+ and Rubinius (though Rubinius does not yet support callbacks).


As an example, building Ruby Extensions under Windows 95/NT is shown as below
Session showing installation and use of FFI in C Ruby:
Building a SWIG extension to Ruby under Windows 95/NT is roughly similar to the process used with Unix.
<pre>
<pre>
C:\swigtest> ruby extconf.rb
$ sudo gem install ffi
C:\swigtest> nmake
Password:
C:\swigtest> nmake install
Building native extensions.  This could take a while...
Successfully installed ffi-0.1.1
1 gem installed
Installing ri documentation for ffi-0.1.1...
Installing RDoc documentation for ffi-0.1.1...
[headius @ cnutter:~]
$ irb
>> require 'ffi'
=> true
>> module RubyFFI
>> extend FFI::Library
>> attach_function :getuid, [], :uint
>> end
=> #<FFI::Invoker:0x1fe8c>
>> puts RubyFFI.getuid
501
=> nil
>>  
</pre>
</pre>


== '''Ruby JAVA extensions''' ==
To extend Ruby with JAVA, the [http://jruby.org/ JRuby] is a good tool to use. JRuby is a high performance, stable, fully threaded Java implementation of the Ruby programming language. It is a Ruby interpreter written entirely in Java and can use Java capabilities from Ruby as well as can use Ruby capabilities from Java.
=== Using JRuby From Command-Line ===
*'''Steps to install'''
– download a binary release [http://jruby.org/ here]


– unzip/untar the downloaded archive
Our hope with JRuby's support of FFI and our release of FFI for C Ruby is that we may finally escape the hell of C extensions.
 
A key feature that's not well documented is the use of FFI's templating system to generate bindings based on the current platform's header files. Here's a sample from the "Etc" module above.


– set JRUBY_HOME environment variable to point to resulting directory
Etc module template, showing how to pull in header files and inspect a struct definition:


– add $JRUBY_HOME/bin to PATH
*'''Steps to use'''
– jruby {script-name}
*'''Example'''
– hello.rb
<pre>
<pre>
name = ARGV[0] || "you"
module Etc
puts "Hello #{name}!"
  class Passwd < FFI::Struct
</pre>
    @@@
– run with
    struct do |s|
<pre>
      s.include "sys/types.h"
jruby hello.rb
      s.include "pwd.h"
</pre>
 
outputs
      s.name "struct passwd"
<pre>
      s.field :pw_name, :string
Hello you!
      s.field :pw_passwd, :string
</pre>
      s.field :pw_uid, :uint
– run with
      s.field :pw_gid, :uint
<pre>
      s.field :pw_dir, :string
jruby hello.rb Mark
      s.field :pw_shell, :string
</pre>
    end
outputs
    @@@
<pre>
  end
Hello Mark
  class Group < FFI::Struct
</pre>
    @@@
=== Using JAVA Classes in JRuby ===
    struct do |s|
To use java classes in JRuby, the program must require 'java'
      s.include "sys/types.h"
      s.include "grp.h"


There several options to provide full names of Java classes to be used in JRuby
      s.name "struct group"
*'''option #1'''
      s.field :gr_name, :string
provide full name when using
      s.field :gr_gid, :uint
<pre>
    end
frame = javax.swing.JFrame.new('My Title')
    @@@
</pre>
  end
*'''option #2'''
assign full class name to a constant
<pre>
JFrame = javax.swing.JFrame
frame = JFrame.new('My Title')
</pre>
*'''option #3'''
use include_class
<pre>
include_class 'javax.swing.JFrame'
frame = JFrame.new('My Title')
</pre>
*'''option #4'''
use include_class with an alias
<pre>
include_class('java.lang.String') do |pkg_name, class_name|
"J#{class_name}"
end
end
msg = JString.new('My Message')
</pre>
</pre>
*'''option #5'''
use include_package
<pre>
module Swing
include_package 'javax.swing'
end
frame = Swing::JFrame.new('My Title')
</pre>
== '''Summary''' ==


== '''See Also''' ==
===Docs on FFI===
*http://java.ociweb.com/mark/NFJS/JRuby.pdf
[http://wiki.jruby.org/wiki/Calling_C_from_JRuby Calling C from JRuby]<br/>
*http://java.ociweb.com/mark/NFJS/RubyCExtensions.pdf
[http://blog.segment7.net/articles/2008/01/15/rubinius-foreign-function-interfaceRubinius's Foreign Function Interface]<br/>
*http://www.swig.org/
[http://pluskid.lifegoo.com/?p=370On the Rubinius FFI]<br/>
*http://www.zenspider.com/ZSS/Products/RubyInline/
 
*http://en.wikibooks.org/wiki/Ruby_Programming/C_Extensions
==Who uses it?==
The libffi library is useful to anyone trying to build a bridge between interpreted and natively compiled code. Some notable users include:
*CPython - the default, most-widely used implementation of the Python programming language uses libffi in the standard ctypes library.
*OpenJDK - the open-source implementation of the Java Platform Standard Edition uses libffi to bridge between the interpreter and native code for some platforms.
*js-ctypes - a foreign function interface for javascript that Mozilla will be shipping in Firefox 3.6.
*Dalvik - Dalvik is the virtual machine which runs the Java platform on Android mobile devices. libffi is used on Android ports for which no custom bridging code has been written.
*Java Native Access (JNA) - the JNI-free way to call native code from Java.
*Ruby-FFI - a Foreign Function Interface extension for Ruby.
*fsbv - Foreign Structure By Value is a foreign function interface library for Common Lisp that extends the standard CFFI package to include support for passing structure arguments by value.
*JSCocoa - call Objective-C code from javascript on Mac OSX and the iPhone (via the libffi-iphone port).
*PyObjC - call Objective-C code from Python on Mac OSX.
*RubyCocoa - call Objective-C code from Ruby on Mac OSX.
*PLT Scheme - call C code from this popular Scheme implementation (also used as the implementation platform for Paul Graham's new Lisp, Arc).
*gcj - the runtime library for the GNU Compiler for the Java Programming Language uses libffi to handle calls back and forth between interpreted and natively compiled code. gcj is part of the GCC, the GNU Compiler Collection.
 
==Supported Platforms==
Libffi has been ported to many different platforms. For specific configuration details and testing status, please refer to the wiki page [http://www.moxielogic.org/wiki/index.php?title=Libffi_3.0.13 here]
 
==Authors and Credits==
 
libffi was originally written by Anthony Green<ref>http://moxielogic.org/blog</ref>
The developers of the GNU Compiler Collection project have made innumerable valuable contributions. See this ChangeLog files in the source distribution for details.
 
Some of the ideas behind libffi were inspired by Gianni Mariani's free gencall library for Silicon Graphics machines.
 
The closure mechanism was designed and implemented by Kresten Krab Thorup.
 
Major processor architecture ports were contributed by the following developers:
 
 
 
　　aarch64          Marcus Shawcroft, James Greenhalgh
 
　　alpha              Richard Henderson
 
　　arm Raffaele Sena
 
　　blackfin        Alexandre Keunecke I. de Mendonca
 
　　cris Simon Posnjak, Hans-Peter Nilsson
 
　　frv Anthony Green
 
　　ia64 Hans Boehm  　
 
　　m32r Kazuhiro Inaoka
 
　　m68k Andreas Schwab
 
　　microblaze Nathan Rossi
 
　　mips Anthony Green, Casey Marshall
 
　　mips64 David Daney
 
　　moxie Anthony Green
 
　　pa Randolph Chung, Dave Anglin, Andreas Tobler
 
　　powerpc Geoffrey Keating, Andreas Tobler,
David Edelsohn, John Hornkvist
 
　　powerpc64 Jakub Jelinek
 
　　s390 Gerhard Tonn, Ulrich Weigand
 
　　sh Kaz Kojima
 
　　sh64 Kaz Kojima
 
　　sparc Anthony Green, Gordon Irlam
 
　　tile-gx/tilepro Walter Lee
 
　　x86 Anthony Green, Jon Beniston
 
　　x86-64 Bo Thorsen
 
　　xtensa Chris Zankel
 
==Adoption==
[http://ruby-doc.org/stdlib-2.0/libdoc/fiddle/rdoc/index.html Fiddle]<br/>
A libffi wrapper in the Ruby Standard Library<br/>
[http://github.com/ffi/ffi Ruby-FFI]<br/>
A Foreign Function Interface extension for Ruby.<br/>
[http://en.wikipedia.org/wiki/RubyCocoa  RubyCocoa]<br/>
Call Objective-C code from Ruby on Mac OSX.<br/>
 
==See Also==
http://en.wikipedia.org/wiki/Libffi
 


== '''References''' ==
== '''References''' ==
<references/>
*http://github.com/atgreen/libffi/blob/master/LICENSE
*http://repository.readscheme.org/ftp/papers/sw2004/barzilay.pdf
*http://hg.mozilla.org/mozilla-central/file/2dc00d4b379a/js/ctypes/libffi
*https://developer.mozilla.org/en/js-ctypes
*https://developer.mozilla.org/en/js-ctypes/js-ctypes_reference
*http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
*http://www.rubyinside.com/ruby-ffi-library-calling-external-libraries-now-easier-1293.html
*http://sourceware.org/libffi/

Revision as of 15:36, 10 February 2014

Ruby libraries to load objects of other languages at run time

This is an advanced topic. Libraries are being developed where Ruby gems can load libraries (and even native code) of other languages.A few open-source projects like FFI are currently working on this.


Introduction

Runtime Library

In computer programming, a runtime library is the API used by a compiler to invoke some of the behaviors of a runtime system. The runtime system implements the execution model and other fundamental behaviors of a programming language. The compiler inserts calls to the runtime library into the executable binary. During execution (run time) of that computer program, execution of those calls to the runtime library cause communication between the application and the runtime system. This often includes functions for input and output, or for memory management. The runtime library may implement a portion of the runtime system's behavior, but if one reads the code of the calls available, they typically are thin wrappers that simply package information and send it to the runtime system. However, sometimes the term runtime library is meant to include the code of the runtime system itself, even though much of that code cannot be directly reached via a library call.

For example, some language features that can be performed only (or are more efficient or accurate) at runtime are implemented in the runtime system and may be invoked via the runtime library API, e.g. some logic errors, array bounds checking, dynamic type checking, exception handling and possibly debugging functionality. For this reason, some programming bugs are not discovered until the program is tested in a "live" environment with real data, despite sophisticated compile-time checking and pre-release testing. In this case, the end user may encounter a runtime error message.

The concept of a runtime library should not be confused with an ordinary program library like that created by an application programmer or delivered by a third party, nor with a dynamic library, meaning a program library linked at run time.

libffi

FFI stands for Foreign Function Interface. A foreign function interface is the popular name for the interface that allows code written in one language to call code written in another language. The libffi library really only provides the lowest, machine dependent layer of a fully featured foreign function interface. A layer must exist above libffi that handles type conversions for values passed between the two languages.<ref>http://sourceware.org/libffi/</ref> libffi is a foreign function interface library. It provides a C programming language interface for calling natively compiled functions given information about the target function at run time instead of compile time. It also implements the opposite functionality: libffi can produce a pointer to a function that can accept and decode any combination of arguments defined at run time. libffi is most often used as a bridging technology between compiled and interpreted language implementations. libffi may also be used to implement plug-ins, where the plug-in's function signatures are not known at the time of creating the host application. Notable users include Python, Haskell, Dalvik, F-Script, PyPy, PyObjC, RubyCocoa, JRuby, Rubinius, MacRuby, gcj, GNU Smalltalk, IcedTea, Cycript, Pawn, Squeak, Java Native Access, Racket,<ref>http://en.wikipedia.org/wiki/Libffi#cite_note-2</ref> Embeddable Common Lisp and Mozilla.<ref>http://en.wikipedia.org/wiki/Libffi#cite_note-3</ref> On Mac OS X libffi is commonly used with BridgeSupport, which provides programming language neutral descriptions of framework interfaces, and Nu which binds direct Objective-C access from Lisp. libffi has been widely ported and is released under a MIT license.

How does it work?<ref>http://www.rubyinside.com/ruby-ffi-library-calling-external-libraries-now-easier-1293.html</ref>

"One of the largest problems plaguing Ruby implementations [..] is the ever-painful story of "extensions". In general, these take the form of a dynamic library, usually written in C, that plugs into and calls Ruby's native API as exposed through ruby.h and libruby."

The many compiled bridges between external libraries and Ruby pose a problem for alternate implementations like JRuby, because of the complexity involved in exposing internals of the implementation or expensive serialization in both directions. Instead, an interface is necessary so that instead of developing libraries that act only as unique bridges to others, we can just have one library that provides the interface to any arbitrary library of our choice.

Ruby already has a library called "dl" that makes it possible to dynamically link external libraries with Ruby. It's a bit arcane though, and Charles points out that it's not widely used "because of real or perceived bugs." Given this, and given the need for an implementation that can be compatible with JRuby, Wayne Meissner has developed "FFI", a new Ruby library that provides "Foreign Function Interface" features to Ruby.

A basic demonstration of a Ruby script that uses C's getpid function should be enough to demonstrate the simplicity of FFI: 

require 'ffi'

module GetPid
  extend FFI::Library
  attach_function :getpid, [], :uint
end

puts GetPid.getpid

Progress of FFI<ref>http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html</ref>

One of the largest problems plaguing Ruby implementations is the ever-painful story of "extensions". In general, these take the form of a dynamic library, usually written in C, that plugs into and calls Ruby's native API as exposed through ruby.h and libruby. Ignoring for the moment the fact that this API exposes way more of Ruby's internals than it should, extensions present a very difficult problem for other implementations:

Do we support them or not?

In many cases, this question is answered for us; most extensions require access to object internals we can't expose, or can't expose without extremely expensive copying back and forth. But there's also a silver lining: the vast majority of C-based extensions exist solely to wrap another library.

Isn't it obvious what's needed here?

This problem has been tackled by a number of libraries on a number of platforms. On the JVM, there's Java Native Access (JNA). On Python, there's ctypes. And even on Ruby, there's the "dl" stdlib, wrapping libdl for programmatic access to dynamic libraries. But dl is not widely used, because of real or perceived bugs and a rather arcane API. Something better is needed.

Wayne is one of the primary maintainers of JNA, and has recently spent time on a new higher-performance version of it called JFFI. Wayne also became a JRuby committer this spring, and perhaps his most impressive contribution to date is a full FFI library for JRuby, based on JNA (eventually JFFI, once we migrate fully) and implementing the full set of what we and Evan agreed would be "FFI API 1.0". We shipped the completed FFI support in JRuby 1.1.4.


The "Passwd" and "Group" structures for functions like 'getpwuid': 

module Etc
  class Passwd < FFI::Struct
    layout :pw_name, :string, 0,
           :pw_passwd, :string, 4,
           :pw_uid, :uint, 8,
           :pw_gid, :uint, 12,
           :pw_dir, :string, 20,
           :pw_shell, :string, 24
  end
  class Group < FFI::Struct
    layout :gr_name, :string, 0,
           :gr_gid, :uint, 8
  end
end

In JRuby 1.1.5, we've taken another step forward with the API, adding support for callbacks. How would we represent a callback you pass into a C function from Ruby?

Binding and calling "qsort" with an array of integers: 

require 'ffi'

module LibC
  extend FFI::Library
  callback :qsort_cmp, [ :pointer, :pointer ], :int
  attach_function :qsort, [ :pointer, :int, :int, :qsort_cmp ], :int
end

p = MemoryPointer.new(:int, 2)
p.put_array_of_int32(0, [ 2, 1 ])
puts "Before qsort #{p.get_array_of_int32(0, 2).join(', ')}"
LibC.qsort(p, 2, 4) do |p1, p2|
  i1 = p1.get_int32(0)
  i2 = p2.get_int32(0)
  i1 < i2 ? -1 : i1 > i2 ? 1 : 0
end
puts "After qsort #{p.get_array_of_int32(0, 2).join(', ')}"

What good is having such a library if it doesn't run everywhere? Up until recently, only Rubinius and JRuby supported FFI, which made our case for cross-implementation use pretty weak. Even though we were getting good use out of FFI, there was no motivation for anyone to use it in general, since the standard Ruby implementation had no support.

That is, until Wayne pulled another rabbit out of his hat and implemented FFI for C Ruby as well. The JRuby team is proud to announce a wholly non-JRuby library: FFI is now available on Ruby 1.9 and Ruby 1.8.6/7, in addition to JRuby 1.1.4+ and Rubinius (though Rubinius does not yet support callbacks).

Session showing installation and use of FFI in C Ruby: 

$ sudo gem install ffi
Password:
Building native extensions.  This could take a while...
Successfully installed ffi-0.1.1
1 gem installed
Installing ri documentation for ffi-0.1.1...
Installing RDoc documentation for ffi-0.1.1...
[headius @ cnutter:~]
$ irb
>> require 'ffi'
=> true
>> module RubyFFI
>> extend FFI::Library
>> attach_function :getuid, [], :uint
>> end
=> #<FFI::Invoker:0x1fe8c>
>> puts RubyFFI.getuid
501
=> nil
>>


Our hope with JRuby's support of FFI and our release of FFI for C Ruby is that we may finally escape the hell of C extensions.

A key feature that's not well documented is the use of FFI's templating system to generate bindings based on the current platform's header files. Here's a sample from the "Etc" module above.

Etc module template, showing how to pull in header files and inspect a struct definition: 

module Etc
  class Passwd < FFI::Struct
    @@@
    struct do |s|
      s.include "sys/types.h"
      s.include "pwd.h"

      s.name "struct passwd"
      s.field :pw_name, :string
      s.field :pw_passwd, :string
      s.field :pw_uid, :uint
      s.field :pw_gid, :uint
      s.field :pw_dir, :string
      s.field :pw_shell, :string
    end
    @@@
  end
  class Group < FFI::Struct
    @@@
    struct do |s|
      s.include "sys/types.h"
      s.include "grp.h"

      s.name "struct group"
      s.field :gr_name, :string
      s.field :gr_gid, :uint
    end
    @@@
  end
end

Docs on FFI

Calling C from JRuby
Foreign Function Interface
the Rubinius FFI

Who uses it?

The libffi library is useful to anyone trying to build a bridge between interpreted and natively compiled code. Some notable users include:

  • CPython - the default, most-widely used implementation of the Python programming language uses libffi in the standard ctypes library.
  • OpenJDK - the open-source implementation of the Java Platform Standard Edition uses libffi to bridge between the interpreter and native code for some platforms.
  • js-ctypes - a foreign function interface for javascript that Mozilla will be shipping in Firefox 3.6.
  • Dalvik - Dalvik is the virtual machine which runs the Java platform on Android mobile devices. libffi is used on Android ports for which no custom bridging code has been written.
  • Java Native Access (JNA) - the JNI-free way to call native code from Java.
  • Ruby-FFI - a Foreign Function Interface extension for Ruby.
  • fsbv - Foreign Structure By Value is a foreign function interface library for Common Lisp that extends the standard CFFI package to include support for passing structure arguments by value.
  • JSCocoa - call Objective-C code from javascript on Mac OSX and the iPhone (via the libffi-iphone port).
  • PyObjC - call Objective-C code from Python on Mac OSX.
  • RubyCocoa - call Objective-C code from Ruby on Mac OSX.
  • PLT Scheme - call C code from this popular Scheme implementation (also used as the implementation platform for Paul Graham's new Lisp, Arc).
  • gcj - the runtime library for the GNU Compiler for the Java Programming Language uses libffi to handle calls back and forth between interpreted and natively compiled code. gcj is part of the GCC, the GNU Compiler Collection.

Supported Platforms

Libffi has been ported to many different platforms. For specific configuration details and testing status, please refer to the wiki page here

Authors and Credits

libffi was originally written by Anthony Green<ref>http://moxielogic.org/blog</ref> The developers of the GNU Compiler Collection project have made innumerable valuable contributions. See this ChangeLog files in the source distribution for details.

Some of the ideas behind libffi were inspired by Gianni Mariani's free gencall library for Silicon Graphics machines.

The closure mechanism was designed and implemented by Kresten Krab Thorup.

Major processor architecture ports were contributed by the following developers:


　　aarch64 Marcus Shawcroft, James Greenhalgh

　　alpha Richard Henderson

　　arm Raffaele Sena

　　blackfin Alexandre Keunecke I. de Mendonca

　　cris Simon Posnjak, Hans-Peter Nilsson

　　frv Anthony Green

　　ia64 Hans Boehm 　

　　m32r Kazuhiro Inaoka

　　m68k Andreas Schwab

　　microblaze Nathan Rossi

　　mips Anthony Green, Casey Marshall

　　mips64 David Daney

　　moxie Anthony Green

　　pa Randolph Chung, Dave Anglin, Andreas Tobler

　　powerpc Geoffrey Keating, Andreas Tobler, David Edelsohn, John Hornkvist

　　powerpc64 Jakub Jelinek

　　s390 Gerhard Tonn, Ulrich Weigand

　　sh Kaz Kojima

　　sh64 Kaz Kojima

　　sparc Anthony Green, Gordon Irlam

　　tile-gx/tilepro Walter Lee

　　x86 Anthony Green, Jon Beniston

　　x86-64 Bo Thorsen

　　xtensa Chris Zankel

Adoption

Fiddle
A libffi wrapper in the Ruby Standard Library
Ruby-FFI
A Foreign Function Interface extension for Ruby.
RubyCocoa
Call Objective-C code from Ruby on Mac OSX.

See Also

http://en.wikipedia.org/wiki/Libffi


References

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2014/ch1_1w1h_jg&oldid=83108"