20th



NAME

Inline - Use other programming languages inside Perl scripts and modules.


SYNOPSIS

    print "9 + 16 = ", add(9, 16), "\n";
    print "9 - 16 = ", subtract(9, 16), "\n";

    use Inline C => <<'END_OF_C_CODE';

    int add(int x, int y) {
      return x + y;
    }

    int subtract(int x, int y) {
      return x - y;
    }

    END_OF_C_CODE


DESCRIPTION

The Inline module allows you to put source code from other programming languages directly ``inline'' in a Perl script or module. The code is automatically compiled as needed, and then loaded for immediate access from Perl.

Inline saves you from the hassle of having to write and compile your own glue code using facilities like XS or SWIG. Simply type the code where you want it and run your Perl as normal. All the hairy details are handled for you. The compilation and installation of your code chunks all happen transparently; all you will notice is the delay of compilation.

The Inline code only gets compiled the first time you run it (or whenever it is modified) so you only take the performance hit once. Code that is Inlined into distributed modules (like on the CPAN) will get compiled when the module is installed, so the end user will never notice the compilation time.

Best of all, it works the same on both Unix and Microsoft Windows. See SUPPORTED PLATFORMS below.

Why Inline?

Do you want to know ``Why would I use other languages in Perl?'' or ``Why should I use Inline to do it?''? I'll try to answer both.

Why would I use other languages in Perl?
The most obvious reason is performance. For an interpreted language, Perl is very fast. Many people will say ``Anything Perl can do, C can do faster''. (They never mention the development time ;-) Anyway, you may be able to remove a bottleneck in your Perl code by using another language, without having to write the entire program in that language. This keeps your overall development time down, because you're using Perl for all of the non-critical code.

Another reason is to access functionality from existing API-s that use the language. Some of this code may only be available in binary form. But by creating small subroutines in the native language, you can ``glue'' existing libraries to your Perl. As a user of the CPAN, you know that code reuse is a good thing. So why throw away those Fortran libraries just yet?

If you are using Inline with the C language (currently the only way to do it), then you can access the full internals of Perl itself. This opens up the floodgates to both extreme power and peril.

Maybe the best reason is ``Because you want to!''. Diversity keeps the world interesting. TMTOWTDI!

Why should I use Inline to do it?
There are already two major facilities for extending Perl with C. They are XS and SWIG. Now if you're familiar with either, then I may be preaching to the choir. Well, here goes:
 <SERMON>

Greetings congregation. This morning I want to open your eyes to the virtues of Inline and the perils of XS. Let us compare the two.

---

Inline - You can use it from a regular script.

XS - Requires you to create a module and an XS file and a makefile, in addition to your regular script. Actually, the program h2xs does a nice job of getting you started, but that's still a lot of junk to maintain.

---

XS - You need rebuild every time you want to test a small change.

Inline - Perl programmers cannot be bothered with silly things like compiling. ``Tweak, Run, Tweak, Run'' is our way of life. Inline does all the dirty work for you.

---

XS - There is a difficult learning curve involved with setting up and using the XS environment. (At least for a simple Perl preacher like me.) Read the following perldocs and man pages if you don't believe me:

 * perlxs
 * perlxstut
 * perlapi
 * perlguts
 * perlmod
 * h2xs
 * xsubpp
 * ExtUtils::MakeMaker

Inline - Makes easy things easy, and hard things possible. Just like Perl.

---

XS - Only implements C and C++.

Inline - Plans to implement several languages. For now, Inline only implements C and it uses XS to do it. (Dirty little secret) But this is the right thing to do. See SUPPORTED LANGUAGES below.

---

Amen.

 </SERMON>

How it works

Inline performs the following steps:

  1. ) Receive the Source Code
    Inline gets the source code from your script or module with a statement like the following:
     use Inline C => Source-Code;

    where C is the programming language of the source code, and Source-Code is a string (most easily represented by using the ``Here Document'' quoting style; see SYNOPSIS above), a file name, an open file handle, or a reference to a subroutine (that will return source code).

    Since Inline is coded in a ``use'' statement, everything is done during Perl's compile time. If anything needs to be done that will affect the Source-Code string, it needs to be done in a BEGIN block that is before the ``use Inline ...'' statement. This might include setting interpolated variables, or setting options in the Inline::Config module.

  2. ) Check if the Source Code has been Compiled
    Inline only needs to compile the source code if it has not yet been compiled. It accomplishes this seemingly magical task in an extremely simple and straightforward manner. It runs the source text through the Digest::MD5 module to produce a 128-bit ``fingerprint'' which is virtually unique. The fingerprint (in hex) is mangled with the current package name (and the script name, if the package is ``main'') along with the name of the programming language, to form a unique name for the executable module. For instance, the C code from examples/example001.pl (see Examples In C) would mangle into:
     main_C_example001_pl_3a9a7ba88a8fb10714be625de5e701f1.so

    If an executable with that name already exists, then proceed to step 8. (No compilation is necessary)

  3. ) Find a Place to Build and Install
    At this point Inline knows it needs to compile the source code. The first thing to figure out is where to create the great big mess associated with compilation, and where to put the object when it's done.

    By default Inline will try to build and install under the first one of the following places that is a valid directory and is writable:

         1) $ENV{PERL_INLINE_BLIB}
            (The PERL_INLINE_BLIB environment variable overrides all else)
         2) ./blib_I/
            (The current directory, unless you're in your home directory)
         3) $bin/blib_I/
            (Where '$bin' is the directory that the script is in)
         4) $ENV{HOME}/blib_I/
            (Under your home directory)
         5) $ENV{HOME}/.blib_I/
            (Same as above but more discrete)

    If none of those exist, Inline will attempt to create and use one of following:

         6) $bin/blib_I/
         7) ./blib_I/

    Failing that, Inline will croak. Optionally, you can configure Inline to build and install exactly where you want, using Inline::Config. See the Inline::Config manpage.

    
    If $Inline::Config::SITE_INSTALL=1, then Inline will only use ./blib_I/ to build in, and the $Config{installsitearch} directory to install the executable in. This option is intended to be used in modules that are to be distributed on the CPAN, so that they get installed in the proper place.

    Optionally, you can configure Inline to build and install exactly where you want.

    NOTE: blib stands for ``build library'' in Perl-speak. It is a directory that gets created when you install a Perl module on your system. blib_I is the Inline.pm version of the same concept.

  4. ) Parse the Source for Semantic Cues
    Inline uses the module Parse::RecDescent to parse through your chunks of source code and look for things that it can create run-time bindings to. For instance, in C it looks for all of the function definitions and breaks them down into names and data types. These elements are used to correctly bind the C function to a Perl subroutine.

  5. ) Create the Build Environment
    Now Inline can take all of the gathered information and create an environment to build your source code into an executable. Without going into all the details, it just creates the appropriate directories, creates the appropriate source files including an XS file and a Makefile.PL.

  6. ) Compile the Code and Install the Executable
    The planets are in alignment. Now for the easy part. Inline just does what you would do to install a module. ``perl Makefile.PL && make && make test && make install''. If something goes awry, Inline will croak with a message indicating where to look for more info.

  7. ) Tidy Up
    By default, Inline will remove all of the mess created by the build process, assuming that everything worked. If the compile fails, Inline will leave everything intact, so that you can debug your errors. Setting $Inline::Config::CLEAN_AFTER_BUILD=0 will also stop Inline from cleaning up.

  8. ) DynaLoad the Executable
    Inline uses the DynaLoader::bootstrap method to pull your external module into Perl space. Now you can call all of your external functions like Perl subroutines. Wheeee!

Another Way To Do It

Instead of specifying the source code as a here-document string, you may want to put it at the end of your script, after the __END__ statement. Then you can pass it to Inline using the DATA filehandle, like this:

    use Inline;
    Inline->import(C => <DATA>);

    print "9 + 16 = ", add(9, 16), "\n";
    print "9 - 16 = ", subtract(9, 16), "\n";

    __END__

    int add(int x, int y) {
      return x + y;
    }

    int subtract(int x, int y) {
      return x - y;
    }

Since the use command is executed at compile time and the DATA filehandle isn't available until runtime, you'll need to invoke the import call manually. This is a useful idiom anytime you need to specify Inline code at runtime.

Configuration

Inline trys to do the right thing as often as possible. But sometimes you may need to override the default actions. This is where Inline::Config comes to the rescue. Inline::Config gives you a more fine-grained control over the entire process. The other side of that coin is ``you need to know what you are doing''.

An important point to remember is that the config settings must be done before the Inline code is evaluated. Since a ``use'' happens at (Perl's) compile time, you may need to something like this:

    BEGIN {
        use Inline;
        $Inline::Config::OPTION_NUMBER_9 = 'Yes';
    # or
        Inline::Config->new->option_number_9('Yes');
    }

    use Inline C => "C code goes here...";

See the Inline::Config manpage for more info.

Configuration from the Command Line

Inline lets you set many of the configuration options from the command line. This can be very handy, especially when you only want to set the options temporarily, for say, debugging.

For instance, to get some general information about your Inline code in the script Foo.pl, use the command:

    perl -MInline=INFO Foo.pl

If you want to force your code to compile, even if its already done, use:

    perl -MInline=FORCE Foo.pl

If you want to do both, use:

    perl -MInline=INFO -MInline=FORCE Foo.pl

or better yet:

    perl -MInline=INFO,FORCE Foo.pl

See the Inline::Config manpage for more info.

Writing Modules with Inline

Writing CPAN modules that use other programming languages is easy with Inline. Let's say that you wanted to write a module called Math::Simple using the previous example code. Start by using the following command:

    h2xs -PAXn Math::Simple

This will generate a bunch of files that form a skeleton of what you need for a distributable module. Next, modify the Simple.pm file to look like this:

    package Math::Simple;

    use strict;
    use vars qw($VERSION @ISA @EXPORT_OK);
    require Exporter;
    @ISA = qw(Exporter);
    @EXPORT_OK = qw(add subtract);
    BEGIN {
        $VERSION = '0.01';
    }

    use Inline;
    Inline->import(C => <DATA>);

    1;

    __DATA__

    int add(int x, int y) {
      return x + y;
    }

    int subtract(int x, int y) {
      return x - y;
    }

Finally, you need to add the following line to the top of your test.pl file:

    use Inline SITE_INSTALL;

When the person installing Math::Simple does a ``make test'', the Inline module will compile the Inlined code and place the executable code into the ./blib directory. Then when a ``make install'' is done, the module will be copied into Perl's $Config{installsitearch} directory (which is where an installed module should go).

Now all you need to do is:

    perl Makefile.PL
    make dist

That will generate the file Math-Simple-0.01.tar.gz which is a distributable package.

Fancy Tricks

The Inline module opens up all sorts of possibilities regarding what you can do with Perl and C. Since everything happens at run time (depending on how you think of it) you can generate C code on the fly and effectively 'eval' it. (How this might be useful is left as an exercise to the reader :-)

Here is how you would code such a beast:

    BEGIN {$c_code = &c_code_generator()}
    use Inline C => $c_code;  # will die if code doesn't compile
    my_function();

or

    $c_code = &c_code_generator();
    eval {use Inline C => $c_code};
    if ($@) {
        handle_error($@);     # trap error if code doesn't compile
    }
    else {
        my_function();
    }


SUPPORTED LANGUAGES

Currently, ``C'' is the only supported language. This is obviously the most important language to support. That is because Perl itself is written in C. By giving a your Perl scripts access to C, you in effect give them access to the entire glorious internals of Perl. (Caveat scriptor :-)

C is also the easiest language to implement because the tools needed to do so, (like XS and ExtUtils::MakeMaker) have already been written and are very flexible and reliable. Inline currently makes use of these pre-existing tools.

But there is definitely no reason why Inline must or should stop with C. As long as sensible bindings can be defined between Perl and another language, that language could be a candidate for the Inline module. Current languages I am considering adding support for include:

 - C++
 - Fortran
 - Pascal
 - Python

Note: Since many C compilers allow the use of assembly code within C, you may want to consider Assembly Language as supported. Ready to start scripting out new device drivers?


SUPPORTED PLATFORMS

This module should work anywhere that CPAN extension modules (those that use XS) can be installed, using the typical install format of:

    perl Makefile.PL
    make
    make test
    make install

It has been tested on many Unix variants and Windows NT.

NOTE: Inline.pm requires Perl 5.005 or higher because Parse::RecDescent requires it. (Something to do with the qr operator)

Inline has been tested on the following platforms:

 V#   OS      OS V#   Perl V# Human              Email
 0.25 Linux   2.2.13  5.00503 Brian Ingerson     ingy@cpan.org
 0.25 Linux   2.2.13  5.6     Brian Ingerson     ingy@cpan.org
 0.20 FreeBSD 3.4     5.00503 Timothy A Gregory  tgregory@tarjema.com
 0.20 FreeBSD 4.0     5.00503 Timothy A Gregory  tgregory@tarjema.com
 0.20 FreeBSD 4.0     5.6     Timothy A Gregory  tgregory@tarjema.com
 0.20 Linux   2.0.36  5.00503 Prakasa Bellam     pbellam@cobaltgroup.com
 0.20 HPUX    B.10.20 5.00503 Jamie Shaffer      jshaffer@chronology.com
 0.20 SunOS   5.6     5.6.0   Jamie Shaffer      jshaffer@chronology.com
 0.20 SunOS   5.5.1   5.6.0   Jamie Shaffer      jshaffer@chronology.com
 0.22 OpenBSD 2.7     5.6.0   Jeremy Devenport   jeremy@weezel.com
 0.22 FreeBSD 3.1     5.00503 Doug Beaver        dougb@scalar.org
 0.25 WinNT   4.0 sp6 5.00503 Brian Ingerson     ingy@cpan.org
 0.24 Cygwin  1.1.1   5.6.0   Leo Schalkwyk      L.Schalkwyk@iop.kcl.ac.uk

The Microsoft tests deserve a little more explanation. I used the following:

 Windows NT 4.0 (service pack 6)
 Perl 5.005_03 (ActiveState build 522)
 MS Visual C++ 6.0
 The "nmake" make utility (distributed w/ Visual C++)

Inline.pm pulls all of its base configuration (including which make utility to use) from config.pm. Since your MSWin32 version of Perl probably came from ActiveState (as a binary distribution) the Config.pm will indicate that nmake is the system's make utility. That is because ActiveState uses Visual C++ to compile Perl.

To install Inline.pm (or any other CPAN module) on MSWin32 w/ Visual C++, use these:

    perl Makefile.PL
    nmake
    nmake test
    nmake install

The ``Cygwin'' test was done on a Windows 98 machine using the Cygwin Unix/Win32 porting layer software from Cygnus. The perl binary on this machine was also compiled using the Cygwin tool set (gcc). This software is freely available from http://sources.redhat.com/cygwin/

If Inline works on your platform, please email me the info above. If it doesn't work, see BUGS AND DEFICIENCIES below.


SEE ALSO

the Inline::Config manpage and the Inline::C::Tutorial manpage


BUGS AND DEFICIENCIES

This is ALPHA code. The interface may still change.

When reporting a bug, please do the following:

 - Put "use Inline REPORTBUG;" at the top of your code, or
   use the command line option "perl -MInline=REPORTBUG ...".
 - Run your code.
 - Follow the printed directions.

Here are some things to watch out for:

  1. The Parse::RecDescent grammar for C is fledgling. It'll get better. For now be careful and examine the generated code when things don't work. Also, using ``perl -MInline=INFO ...'' will give you useful information.

  2. Inline doesn't yet support custom typemapping. To pass anything beyond basic C types, use the type SV* and do your own conversions ``inline''. See the Inline::C::Tutorial manpage for more information on programming with C.

  3. While Inline does attempt to clean up after itself, there is currently no functionality to remove a shared object when a new version is compiled. This shouldn't be hard to do, but I want to think about it a little more.

  4. The compile time using Visual C++ on MSWin32 seems to be much slower in my tests so far. During this time, your script will seem to hang. Just be patient. After compilation, the execution time is comparable.


AUTHOR

Brian Ingerson <INGY@cpan.org>


COPYRIGHT

Copyright (c) 2000, Brian Ingerson. All Rights Reserved. This module is free software. It may be used, redistributed and/or modified under the terms of the Perl Artistic License.

(see http://www.perl.com/perl/misc/Artistic.html)