Appendix D. Perlbrew, CPAN, and cpanm

Written by Alastair McGowan-Douglas

Perl comes preinstalled on UNIX systems. Normally, you will find that the preinstalled version is older than the latest Perl release, and in some cases it will be subtly different. This Perl is designed by the system maintainers to work with the rest of the system. When you install packages, such as with the system package manager, the packages you install will have been tested against the version of Perl installed by the system.

Many of those packages are precompiled and tested versions of modules available on CPAN, the Comprehensive Perl Archive Network. This allows you to use those modules and that version of Perl to write your own scripts and applications on your system.

This is fine, until you either want to upgrade Perl or install modules that don’t exist in your package manager. It is worthwhile to upgrade Perl so as to be up to date with the latest bug fixes and features. Installing modules from CPAN is a cornerstone of Perl programming; if you can’t get things from CPAN, you’ll find yourself reinventing wheels, writing nonportable code, and unable to use scripts others have written that rely on CPAN modules.

Upgrading and installing are two separate processes, and they are solved in two different ways, but they are related.

1. At the time of writing, the traditional CPAN search site (http://search.cpan.org) is fluctuating between available and unavailable. That site is being phased out for not being community-run—the code is not available and the community cannot access the server that hosts it. The new site, metacpan, is available on github at https://github.com/CPAN-API.

Click here to view code image

use Module;
Can't locate Module.pm in @INC (you may need to install the Module module)

(This particular error message was introduced in 5.18; prior to this version, it did not give the hint about installing the module.)

The use directive defines a requirement; semantically, it says that the script in question cannot run without the stated module. Hence, Perl refuses to go further without the module. The message states that the file could not be found in @INC, which you will recognize as the name of a Perl array. @INC is, indeed, an array; the array that contains all the search paths for modules.

You can inspect the contents of @INC by either using it as a normal array and printing them, or asking perl -V to list them:

Click here to view code image

$ perl -E'say for @INC'²
/etc/perl
/usr/local/lib/perl/5.18.2
/usr/local/share/perl/5.18.2
        ...
        <snip>

$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 2) configuration:

  Platform:
    osname=linux, osvers=3.2.0-58-generic, archname=x86_64-linux-
gnu-thread-multi
       ...
       <snip>
       ...
 Compiled at Mar 27 2014 18:30:28
  @INC:
    /etc/perl
    /usr/local/lib/perl/5.18.2
    /usr/local/share/perl/5.18.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.18
    /usr/share/perl/5.18
    /usr/local/lib/site_perl
    .

2. -E is a new option introduced with 5.10 that is equivalent to -e’use feature “:5.10”; -E turns on all the features introduced with 5.10. Here, we use it, in order to use say.

These are all the directories Perl will look in by default.

Listed is the default output for the Perl provided with Linux Mint 17, which comes with 5.18; your output may differ.

These directories are the places Perl looks when your script requests a module. Apart from ‘.’, however, all of these directories are system directories. Only root can write to these. Some of these directories don’t even exist! This is just a default set that Perl is built with. The ones that do exist are:

/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.18
/usr/share/perl/5.18
.

(Of course the current directory exists!) Files in the lib directories are architecture-dependent, and therefore are going files that interface between Perl and existing C libraries, or else are written in C for efficiency purposes. Those in share are everything else—pure Perl modules, essentially.

Keep in mind, that all of these files were provided by apt, Debian’s package manager. None of them were installed manually by the user. What if you want to use a module from CPAN, but apt hasn’t installed it?

Less often, you will have been provided a module from somewhere other than CPAN. This can happen if you are using code from a third-party vendor. This is very uncommon. Most modules written by third parties go on CPAN anyway, because they interface into proprietary software or services, meaning the company suffers nothing from making the modules themselves free on CPAN.

In all cases, you can tell Perl where to look for modules using the -I option to Perl, which prepends the @INC array with the requested directory. (Ordinarily you will have a distribution with a script and a lib directory containing the script’s modules). The -I option is shown here:

$ perl -I lib example.pl

This is the technique cpan will use when testing modules prior to installation. With -I, the temporary directory with the module in it can be added to @INC, and the test scripts run in this context. This ensures the module being tested is the one in the temporary directory, and not an older version installed in system Perl.

For more permanent locations for your modules, you can use the PERL5LIB environment variable. Simply set it to a set of colon-separated paths, much like PATH itself, and these will be added to @INC as well:

Click here to view code image

$ PERL5LIB="$HOME/perl5:$PERL5LIB" perl example.pl

Modules from the package manager have the advantage of having been tested by the vendor; i.e., the people who put the operating system together in the first place. You will often hear the term “vendor Perl” used to refer to the system Perl that your vendor supplies. Additionally, on many systems, the packaged versions are precompiled. This means the procedure for getting a Perl module with C dependencies is a simple matter of unpacking it. Later, we will see how otherwise complex this can be.

This method is not without its drawbacks, however, including the following:

• You need root access. If you are working in a corporate environment, you are unlikely to have this access. Sometimes there are procedures in place to request that those who do have access install it for you, but these processes take time and aren’t necessarily going to be granted.

• It installs to system Perl. If you are using Perlbrew (discussed later), you will not be able to use a prepackaged version, because Perlbrew creates custom Perl installations independent of vendor Perl.

• The version in the package manager might not be the version containing the features you want. You will have to get the latest version from CPAN if the packaged version is older.

• It might not be there! There are tens of thousands of distributions on CPAN, and it is infeasible for a vendor to package, test, and distribute every one of them. With the recent resurgence of Perl’s popularity (the “Modern Perl” movement) CPAN’s popularity has had an equivalent increase in activity. That means that, in many cases, CPAN is gaining new and updated distributions faster than vendors can keep up. Vendors, therefore, have to pick a snapshot of CPAN, compile a popular subset of the modules, and distribute that.

• You can have anything.

• You don’t need root.

• You can install to a custom Perl.

• You need a C compiler, and C dependencies.

• It hasn’t been thoroughly tested for your particular situation.

Installation of modules from CPAN is done with the cpan command that comes with Perl, but it is recommended to fetch and install cpanm instead. The cpan command is usable but requires setup; while cpanm comes with defaults for everything and tries to be as DWIM (Do What I Mean) as possible.

The first thing to realize is that, by default, cpan is going to try to install to system Perl directories. This is because cpan is a Perl script itself, which means that system Perl will be used to run it! And that requires root access. To avoid that, we will use local::lib to create and use a local lib/ for Perl modules.

The following five environment variables are all you will need to create a local Perl setup. With them in effect, cpan can work with any directory on the system.

Traditionally, your $HOME will be used to contain modules and scripts you want to install without root. This is known as local Perl (as opposed to system, or vendor, Perl). Another common location is site Perl, which generally refers to that set of modules your organization commonly uses, and hence is installed on all servers by default. For our purposes, we’ll be dealing with the most common case, which is to install a local Perl library (a local lib) in your home directory.

Listed here are the five environment variables:

Click here to view code image

PATH="/home/altreus/perl5/bin${PATH+:}$PATH"
PERL5LIB="/home/altreus/perl5/lib/perl5${PERL5LIB+:}$PERL5LIB"
PERL_LOCAL_LIB_ROOT="/home/altreus/perl5${PERL_LOCAL_LIB_ROOT+:}$PERL_
LOCAL_LIB_ROOT"
PERL_MB_OPT="--install_base "/home/altreus/perl5""
PERL_MM_OPT="INSTALL_BASE=/home/altreus/perl5"

The five environment variables listed here all contain the same directory: /home/altreus/perl5. This is the local Perl directory. You can copy the five environment variables and amend them for your own purposes, or you can use the local::lib module.

Unfortunately, you need to install local::lib before you can use it to install modules! local::lib can usually be installed from package managers, but it is possible to install it to local Perl before you have it in the first place.

https://metacpan.org/pod/local::lib is the de facto source for local::lib. It contains a section, “The bootstrapping technique,” that explains how to install it up front. Essentially it is as follows:

Click here to view code image

$  wget  http://cpan.metacpan.org/authors/id/H/HA/HAARG/local-lib-2.000012.
tar.gz
$ tar xzf local-lib-2.000012.tar.gz
$ cd local-lib-2.000012
$ perl Makefile.PL --bootstrap
$ make test && make install

The documentation then tells you how to set up your environment for the future:

Click here to view code image

$ echo '[ $SHLVL -eq 1 ] && eval "$(perl -I$HOME/perl5/lib/perl5
-Mlocal::lib)"' >>~/.bashrc

The preceding command assumes you are using bash as your shell, which is the default shell for most UNIX systems, including Mac OS X. (Users of other shells will need to rewrite the command to adapt to their particular shell syntax.) All the procedure does is automate the exporting of the environment variables mentioned earlier.

Remember to start a new shell if you change your .bashrc, so that the new environment variables are loaded; or just source your .bashrc. The source builtin command works for bash but may be a different command for other shells.

If you don’t have cpanm, go to http://cpanmin.us and follow the directions given in the provided script comment section. In keeping with the theme of putting everything in your local lib, it is advised to install your own cpanm, even if you do already have it.

Click here to view code image

$ curl -L http://cpanmin.us | perl – App::cpanminus

However, this can cause problems if your connection breaks while installing. Instead, save the file as cpanm.pl, and then run

Click here to view code image

$ perl cpanm.pl App::cpanminus

This simply uses a temporary cpanm to install a permanent cpanm. It will use the local::lib we already set up, so you will find cpanm in ~/perl5/bin by default.

Now that we have both local::lib and cpanm set up, installing a module is a simple matter of running

$ cpanm Module::Name

You can also use the --sudo option if you want to install to site Perl (that’s the root-owned area that is not system Perl), but this is not recommended, because it is not guaranteed such a directory is set up. Remember, you can use local::lib to specify any directory, so you could easily have a site Perl set up with local::lib and then use the sudo option so cpanm can install to it.

With these tools, you can avoid system Perl entirely; except in one aspect, which is the perl runtime itself. For that, we can use Perlbrew.

Different Perl versions have different requirements when installing modules. That means that modules installed under 5.16 might not be compatible with 5.18 or 5.20. We need to keep these installations separate, so that the install directory is relative to the current Perl version, and we avoid conflicts.

We solve all of this with Perlbrew. This will download and compile any version of Perl currently available in its repository (they’re actually on CPAN) and install it into local Perl; which means the whole thing can be done with only user access to the system. (In the next section, we will discuss caveats to this.) We can install as many Perl versions as we like, and simply ask Perlbrew to switch to the one we want to use. That way, perl is dependent on our environment, which is what we want; and because the environment changes, cpanm knows where to install the modules to avoid conflict.

To install Perlbrew, go to http://perlbrew.pl/ and follow the instructions:

Click here to view code image

$ curl -L http://install.perlbrew.pl | bash

This installs Perlbrew locally and initializes it. The output says to append the following piece of code to the end of your ~/.bash_profile and start a new shell. Perlbrew should be up and fully functional from there:

Click here to view code image

    source ~/perl5/perlbrew/etc/bashrc

Depending on your setup, this might have to go into ~/.bashrc instead. Perlbrew will also detect other shells and advise on those as relevant. Note that you should do this instead of the line that local::lib uses. Restart your terminal session if you had to switch this out; this bashrc detects when you’re running with a brewed Perl and sets up a local::lib for you.

With that in your shell’s RC, you can now compile your own Perl in your home directory, and use that in place of system Perl.

$ perlbrew available
  perl-5.21.1
  perl-5.20.0
  perl-5.18.2
      ...
       <snip>

You will need the following line to use Perlbrew properly. It simply assists with installing certain versions of Perl, and it’s easier if it’s just installed always.

$ perlbrew install-patchperl

Finally, you can install any version of Perl listed. The perl- part is optional.

Click here to view code image

$ perlbrew install 5.20.0

       <snip>

$ perlbrew use 5.20.0
$ perl –V
       <snip>
       ...
 @INC:
    /home/altreus/perl5/perlbrew/perls/perl-5.20.0/lib/site_perl/5.20.0/
x86_64-linux
    /home/altreus/perl5/perlbrew/perls/perl-5.20.0/lib/site_perl/5.20.0
    /home/altreus/perl5/perlbrew/perls/perl-5.20.0/lib/5.20.0/x86_64-linux
    /home/altreus/perl5/perlbrew/perls/perl-5.20.0/lib/5.20.0

You can see that your @INC contains only a local lib for 5.20.0. That means Perl 5.20 will not look in system Perl for modules, because they are unlikely to be compatible. However, the installation process for 5.20 will fetch all the core modules as well, so the directories just listed will not be empty.

Perlbrew can also be used to install cpanm locally:

$ perlbrew install-cpanm

This cpanm will always be available, even if you switch out Perl version. That avoids any issue with cpanm having been installed into a local lib that has suddenly changed.

On Mac OS X, you can install Xcode and the command-line tools for it to get a C compiler. On Windows, the C compiler is provided with Strawberry Perl.

It is not just Perl itself that requires a C compiler. Many modules on CPAN are interfaces into C libraries (XML::LibXML is an example), and some modules are written in C for efficiency (JSON has a version like this). Perl modules that use C in some way are known as XS modules. XS is the tool used to interface between Perl and C, allowing a normal .pm to wrap up code written in C and expose the interface to Perl. That’s why the JSON module has two versions: JSON::PP (“Pure Perl”) and JSON::XS.

Interfaces like JSON can be installed without a C compiler because the XS version simply is not installed at all if the C compiler is not available at the time. Some modules are XS but don’t have a pure Perl equivalent; these ones will require a C compiler. But the tricky ones are those that interface into an external library, like XML::LibXML.

XML::LibXML is a Perl interface into the C library libxml2. In order to compile, the development headers for libxml2 have to be available to the C compiler. Exactly how C compiles against external libraries is out of the scope of this discussion, but the difficulty arrives in just how to get these third-party dependencies in the first place. The CPAN toolchain (cpanm, Module::Build, and friends) allows us to list an unlimited chain of dependencies from CPAN itself, but there is no mechanism to list dependencies outside of CPAN. The best we can do is to try to give the user as helpful a message as possible when trying to compile an XS module that interfaces a C library like this.

In the case of XML::LibXML, we need the libxml2 development headers. On Debian, this is the libxml2-dev package. On OS X, Xcode comes with libxml2, but homebrew is a common tool for installing various packages on OS X. Dependencies that don’t come with Xcode are likely to be available with homebrew.

When it comes to C dependencies, then, it is often necessary to install a package. This often requires root access. It is possible to set up the environment so that all the C dependencies are discoverable by the C compiler in $HOME, similar to how Perl itself discovers modules somewhere in $HOME. This can be achieved with the LIBRARY_PATH environment variable. Normally, when developing, the developer has root access and can avoid this issue.

So, the requirement for root can be avoided in general, but the Perl toolchain (cpanm, perlbrew) can only help with Perl itself. Any dependencies outside of Perl will have to be dealt with independently of this.

For those modules not available, Strawberry comes with cpanm by default (since 5.14), and ships with a C compiler.

Windows being mostly a single-user environment, those external dependencies required for compiling modules that use C libraries should be easy to install by locating the Windows installer for that library, installing it, and then trying again. Problems with not having root access don’t tend to occur on Windows.

It is possible to have a similar system to the UNIX method, whereby the library is installed to the user’s home directory, but it is so rare that this is needed that you are unlikely to find a case where a user has been unable to run their script on Windows due to an external C dependency they cannot resolve.

Visit http://strawberryperl.com for Strawberry Perl; you can find a full set of release notes at the bottom of the page, listing the distributions shipped with it. Strawberry Perl contains all modules required to run examples in this book.