Before we get started, you should make a minor change to your copy of CGI.pm. CGI.pm handles HTTP file uploads and automatically saves the contents of these uploads to temporary files. This is a very convenient feature, and we’ll talk about this later. However, file uploads are enabled by default in CGI.pm, and it does not impose any limitations on the size of files it will accept. Thus, it is possible for someone to upload multiple large files to your web server and fill up your disk.

Clearly, the vast majority of your CGI scripts do not accept file uploads. Thus, you should disable this feature and enable it only in those scripts where you wish to use it. You may also wish to limit the size of POST requests, which includes file uploads as well as standard forms submitted via the POST method.

To make these changes, locate CGI.pm in your Perl libraries and then search for text that looks like the following:

# Set this to a positive value to limit the size of a POSTing
# to a certain number of bytes:
$POST_MAX = -1;

# Change this to 1 to disable uploads entirely:
$DISABLE_UPLOADS = 0;

Set $DISABLE_UPLOADS to 1. You may wish to set $POST_MAX to a reasonable upper bound as well, such as 100KB. POST requests that are not file uploads are processed in memory, so restricting the size of POST requests avoids someone submitting multiple large POST requests that quickly use up available memory on your server. The result looks like this:

# Set this to a positive value to limit the size of a POSTing
# to a certain number of bytes:
$POST_MAX = 102_400;  # 100 KB

# Change this to 1 to disable uploads entirely:
$DISABLE_UPLOADS = 1;

If you then want to enable uploads and/or allow a greater size for POST requests, you can override these values in your script by setting $CGI::DISABLE_UPLOADS and $CGI::POST_MAX after you use the CGI.pm module, but before you create a CGI.pm object. We will look at how to receive file uploads later in this chapter.

You may need special permission to update your CGI.pm file. If your system administrator for some reason will not make these changes, then you must disable file uploads and limit POST requests on a script by script basis. Your scripts should begin like this:

#!/usr/bin/perl -wT

use strict;
use CGI;

$CGI::DISABLE_UPLOADS = 1;
$CGI::POST_MAX        = 102_400; # 100 KB

my $q = new CGI;
.
.

Throughout our examples, we will assume that the module has been patched and omit these lines.

CGI.pm is a big module. It provides functions for accessing CGI environment variables and printing outgoing headers. It automatically interprets form data submitted via POST, via GET, and handles multipart-encoded file uploads. It provides many utility functions to do common CGI-related tasks, and it provides a simple interface for outputting HTML. This interface does not eliminate the need to understand HTML, but it makes including HTML inside a Perl script more natural and easier to validate.

Because CGI.pm is so large, some people consider it bloated and complain that it wastes memory. In fact, it uses many creative ways to increase the efficiency of CGI.pm including a custom implementation of SelfLoader. This means that it loads only code that you need. If you use CGI.pm only to parse input, but do not use it to produce HTML, then CGI.pm does not load the code for producing HTML.

There have also been some alternative, lightweight CGI modules written. One of the lightweight alternatives to CGI.pm was begun by David James; he got together with Lincoln Stein and the result is a new and improved version of CGI.pm that is even smaller, faster, and more modular than the original. It should be available as CGI.pm 3.0 by the time you read this book.

CGI.pm, like Perl, is powerful yet flexible. It supports two styles of usage: a standard interface and an object-oriented interface. Internally, it is a fully object-oriented module. Not all Perl programmers are comfortable with object-oriented notation, however, so those developers can instead request that CGI.pm make its subroutines available for the developer to call directly.

Here is an example. The object-oriented syntax looks like this:

use strict;
use CGI;

my $q    = new CGI;
my $name = $q->param( "name" );

print $q->header( "text/html" ),
      $q->start_html( "Welcome" ),
      $q->p( "Hi $name!" ),
      $q->end_html;

The standard syntax looks like this:

use strict;
use CGI qw( :standard );

my $name = param( "name" );

print header( "text/html" ),
      start_html( "Welcome" ),
      p( "Hi $name!" ),
      end_html;

Don’t worry about the details of what the code does right now; we will cover all of it during this chapter. The important thing to notice is the different syntax. The first script creates a CGI.pm object and stores it in $q ($q is short for query and is a common convention for CGI.pm objects, although $cgi is used sometimes, too). Thereafter, all the CGI.pm functions are preceded by $q->. The second asks CGI.pm to export the standard functions and simply uses them directly. CGI.pm provides several predefined groups of functions, like :standard , that can be exported into your CGI script.

The standard CGI.pm syntax certainly has less noise. It doesn’t have all those $q-> prefixes. Aesthetics aside, however, there are good arguments for using the object oriented syntax with CGI.pm.

Exporting functions has its costs. Perl maintains a separate namespace for different chunks of code referred to as packages. Most modules, like CGI.pm, load themselves into their own package. Thus, the functions and variables that modules see are different from the modules and variables you see in your scripts. This is a good thing, because it prevents collisions between variables and functions in different packages that happen to have the same name. When a module exports symbols (whether they are variables or functions), Perl has to create and maintain an alias of each of the these symbols in your program’s namespace, the main namespace. These aliases consume memory. This memory usage becomes especially critical if you decide to use your CGI scripts with FastCGI or mod_perl.

The object-oriented syntax also helps you avoid any possible collisions that would occur if you create a subroutine with the same name as one of CGI.pm’s exported subroutines. Also, from a maintenance standpoint, it is clear from looking at the object-oriented script where the code for the header function is: it’s a method of a CGI.pm object, so it must be in the CGI.pm module (or one of its associated modules). Knowing where to look for the header function in the second example is much more difficult, especially if your CGI scripts grow large and complex.

Some people avoid the object-oriented syntax because they believe it is slower. In Perl, methods typically are slower than functions. However, CGI.pm is truly an object-oriented module at heart, and in order to provide the function syntax, it must do some fancy footwork to manage an object for you internally. Thus with CGI.pm, the object-oriented syntax is not any slower than the function syntax. In fact, it can be slightly faster.

We will use CGI.pm’s object-oriented syntax in most of our examples.

CGI.pm provides many methods to get information about your environment. Of course, when you use CGI.pm, all of your standard CGI environment variables are still available in Perl’s %ENV hash, but CGI.pm also makes most of these available via method calls. It also provides some unique methods. Table 5.1 shows how CGI.pm’s functions correspond to the standard CGI environment variables.

Most of these CGI.pm methods take no arguments and return that same value as the corresponding environment variable. For example, to get the additional path information passed to your CGI script, you can use the following method:

my $path = $q->path_info;

This is the same information that you could also get this way:

my $path = $ENV{PATH_INFO};

However, a few methods differ or have features worth noting. Let’s take a look at these.

if ( $q->Accept( "image/png" ) ) {
    .
    .
    .

my @accept = $q->Accept;
if ( grep $_ eq "image/png", @accept ) {
    .
    .
    .

#!/usr/bin/perl -wT

use strict;
use CGI;

my $q = new CGI;
print $q->header( "text/plain" );

print "These are the HTTP environment variables I received:

";

foreach ( $q->http ) {
    print "$_:
";
    print "  ", $q->http( $_ ), "
";
}

param is probably the most useful method CGI.pm provides. It allows you to access the parameters submitted to your CGI script, whether these parameters come to you via a GET request or a POST request. If you call param without arguments, it will return a list of all of the parameter names your script received. If you provide a single argument to it, it will return the value for the parameter with that name. If no parameter with that name was submitted to your script, it returns undef.

It is possible for your CGI script to receive multiple values for a parameter with the same name. This happens when you create two form elements with the same name or you have a select box that allows multiple selections. In this case, param returns a list of all of the values if it is called in a list context and just the first value if it is called in a scalar context. This may sound a little complicated, but in practice it works such that you should end up with what you expect. If you ask param for one value, you will get one value (even if other values were also submitted), and if you ask it for a list, you will always get a list (even if the list contains only one element).

Example 5.1 is a simple example that displays all the parameters your script receives.

#!/usr/bin/perl -wT

use strict;
use CGI;

my $q = new CGI;
print $q->header( "text/plain" );

print "These are the parameters I received:

";

my( $name, $value );

foreach $name ( $q->param ) {
    print "$name:
";
    foreach $value ( $q->param( $name ) ) {
        print "  $value
";
    }
}

If you call this CGI script with multiple parameters, like this:

http://localhost/cgi/param_list.cgi?color=red&color=blue&shade=dark

you will get the following output:

These are the parameters I received:

color:
  red
  blue
shade:
  dark

$q->param( title => "Web Developer" );

$q->param( hobbies => "Biking", "Windsurfing", "Music" );

$q->delete( "age" );

$q->delete_all;

if ($meth eq 'POST') {
    $self->read_from_client(*STDIN,$query_string,$content_length,0)
        if $content_length > 0;
    # Some people want to have their cake and eat it too!
    # Uncomment this line to have the contents of the query string
    # APPENDED to the POST data.
    # $query_string .= (length($query_string) ? '&' : '') . $ENV{'QUERY_STRING'}
             if defined $ENV{'QUERY_STRING'};
    last METHOD;
}

http://www.localhost.com/cgi/lookup.cgi?cgi+perl

my @words = $q->keywords;            # these lines do the same thing
my @words = $q->param( "keywords" );

foreach ( $q->param ) {
    $q->param( $1, 1 ) if /(.*).x/;
}

One of the problems with using a method to retrieve the value of a parameter is that it is more work to embed the value in a string. If you wish to print the value of someone’s input, you can use an intermediate variable:

my $name = $q->param( 'user' );
print "Hi, $user!";

Another way to do this is via an odd Perl construct that forces the subroutine to be evaluated as part of an anonymous list:

print "Hi, @{[ $q->param( 'user' ) ]}!";

The first solution is more work and the second can be hard to read. Fortunately, there is a better way. If you know that you are going to need to refer to many output values in a string, you can import all the parameters as variables to a specified namespace:

$q->import_names( "Q" );
print "Hi, $Q::user!";

Parameters with multiple values become arrays in the new namespace, and any characters in a parameter name other than a letter or number become underscores. You must provide a namespace and cannot pass “main”, the default namespace, because that might create security risks.

The price you pay for this convenience is increased memory usage because Perl must create an alias for each parameter.

As we mentioned in the last chapter, it is possible to create a form with a multipart/form-data media type that permits users to upload files via HTTP. We avoided discussing how to handle this type of input then because handling file uploads properly can be quite complex. Fortunately, there’s no need for us to do this because, like other form input, CGI.pm provides a very simple interface for handling file uploads.

You can access the name of an uploaded file with the param method, just like the value of any other form element. For example, if your CGI script were receiving input from the following HTML form:

<FORM ACTION="/cgi/upload.cgi" METHOD="POST" ENCTYPE="multipart/form-data">
  <P>Please choose a file to upload:
  <INPUT TYPE="FILE" NAME="file">
  <INPUT TYPE="SUBMIT">
</FORM>

then you could get the name of the uploaded file this way, by referring to the name of the <FILE> input element, in this case “file”:

my $file = $q->param( "file" );

The name you receive from this parameter is the name of the file as it appeared on the user’s machine when they uploaded it. CGI.pm stores the file as a temporary file on your system, but the name of this temporary file does not correspond to the name you get from this parameter. We will see how to access the temporary file in a moment.

The name supplied by this parameter varies according to platform and browser. Some systems supply just the name of the uploaded file; others supply the entire path of the file on the user’s machine. Because path delimiters also vary between systems, it can be a challenge determining the name of the file. The following command appears to work for Windows, Macintosh, and Unix-compatible systems:

my( $file ) = $q->param( "file" ) =~ m|([^/:\]+)$|;

However, it may strip parts of filenames, since “report 11/3/99” is a valid filename on Macintosh systems and the above command would in this case set $file to “99”. Another solution is to replace any characters other than letters, digits, underscores, dashes, and periods with underscores and prevent any files from beginning with periods or dashes:

my $file = $q->param( "file" );
$file =~ s/([^w.-])/_/g;
$file =~ s/^[-.]+//;

The problem with this is that Netscape’s browsers on Windows sends the full path to the file as the filename. Thus, $file may be set to something long and ugly like “C_ _ _Windows_Favorites_report.doc”.

You could try to sort out the behaviors of the different operating systems and browsers, check for the user’s browser and operating system, and then treat the filename appropriately, but that would be a very poor solution. You are bound to miss some combinations, you would constantly need to update it, and one of the greatest advantages of the Web is that it works across platforms; you should not build any limitations into your solutions.

So the simple, obvious solution is actually nontechnical. If you do need to know the name of the uploaded file, just add another text field to the form allowing the user to enter the name of the file they are uploading. This has the added advantage of allowing a user to provide a different name than the file has, if appropriate. The HTML form looks like this:

<FORM ACTION="/cgi/upload.cgi" METHOD="POST" ENCTYPE="multipart/form-data">
  <P>Please choose a file to upload:
  <INPUT TYPE="FILE" NAME="file">
  <P>Please enter the name of this file:
  <INPUT TYPE="TEXT" NAME="filename">
</FORM>

You can then get the name from the text field, remembering to strip out any odd characters:

my $filename = $q->param( "filename" );
$filename =~ s/([^w.-])/_/g;
$filename =~ s/^[-.]+//;

So now that we know how to get the name of the file uploaded, let’s look at how we get at the content. CGI.pm creates a temporary file to store the contents of the upload; you can get a file handle for this file by passing the name of the file according to the file element to the upload method as follows:

my $file = $q->param( "file" );
my $fh   = $q->upload( $file );

The upload method was added to CGI.pm in Version 2.47. Prior to this you could use the value returned by param (in this case $file) as a file handle in order to read from the file; if you use it as a string it returns the name of the file. This actually still works, but there are conflicts with strict mode and other problems, so upload is the preferred way to get a file handle now. Be sure that you pass upload the name of the file according to param, and not a different name (e.g., the name the user supplied, the name with nonalphanumeric characters replaced with underscores, etc.).

Note that transfer errors are much more common with file uploads than with other forms of input. If the user presses the Stop button in the browser as the file is uploading, for example, CGI.pm will receive only a portion of the uploaded file. Because of the format of multipart/form-data requests, CGI.pm will recognize that the transfer is incomplete. You can check for errors such as this by using the cgi_error method after creating a CGI.pm object. It returns the HTTP status code and message corresponding to the error, if applicable, or an empty string if no error has occurred. For instance, if the Content-length of a POST request exceeds $CGI::POST_MAX, then cgi_error will return “413 Request entity too large”. As a general rule, you should always check for an error when you are recording input on the server. This includes file uploads and other POST requests. It doesn’t hurt to check for an error with GET requests either.

Example 5.2 provides the complete code, with error checking, to receive a file upload via our previous HTML form.

#!/usr/bin/perl -wT

use strict;
use CGI;
use Fcntl qw( :DEFAULT :flock );

use constant UPLOAD_DIR     => "/usr/local/apache/data/uploads";
use constant BUFFER_SIZE    => 16_384;
use constant MAX_FILE_SIZE  => 1_048_576;       # Limit each upload to 1 MB
use constant MAX_DIR_SIZE   => 100 * 1_048_576; # Limit total uploads to 100 MB
use constant MAX_OPEN_TRIES => 100;

$CGI::DISABLE_UPLOADS   = 0;
$CGI::POST_MAX          = MAX_FILE_SIZE;

my $q = new CGI;
$q->cgi_error and error( $q, "Error transferring file: " . $q->cgi_error );

my $file      = $q->param( "file" )     || error( $q, "No file received." );
my $filename  = $q->param( "filename" ) || error( $q, "No filename entered." );
my $fh        = $q->upload( $file );
my $buffer    = "";

if ( dir_size( UPLOAD_DIR ) + $ENV{CONTENT_LENGTH} > MAX_DIR_SIZE ) {
    error( $q, "Upload directory is full." );
}

# Allow letters, digits, periods, underscores, dashes
# Convert anything else to an underscore
$filename =~ s/[^w.-]/_/g;
if ( $filename =~ /^(w[w.-]*)/ ) {
    $filename = $1;
}
else {
    error( $q, "Invalid file name; files must start with a letter or number." );
}

# Open output file, making sure the name is unique
until ( sysopen OUTPUT, UPLOAD_DIR . $filename, O_CREAT | O_EXCL ) {
    $filename =~ s/(d*)(.w+)$/($1||0) + 1 . $2/e;
    $1 >= MAX_OPEN_TRIES and error( $q, "Unable to save your file." );
}

# This is necessary for non-Unix systems; does nothing on Unix
binmode $fh;
binmode OUTPUT;

# Write contents to output file
while ( read( $fh, $buffer, BUFFER_SIZE ) ) {
    print OUTPUT $buffer;
}

close OUTPUT;


sub dir_size {
    my $dir = shift;
    my $dir_size = 0;
    
    # Loop through files and sum the sizes; doesn't descend down subdirs
    opendir DIR, $dir or die "Unable to open $dir: $!";
    while ( readdir DIR ) {
        $dir_size += -s "$dir/$_";
    }
    return $dir_size;
}


sub error {
    my( $q, $reason ) = @_;
    
    print $q->header( "text/html" ),
          $q->start_html( "Error" ),
          $q->h1( "Error" ),
          $q->p( "Your upload was not procesed because the following error ",
                 "occured: " ),
          $q->p( $q->i( $reason ) ),
          $q->end_html;
    exit;
}

We start by creating several constants to configure this script. UPLOAD_DIR is the path to the directory where we will store uploaded files. BUFFER_SIZE is the amount of data to read into memory while transferring from the temporary file to the output file. MAX_FILE_SIZE is the maximum file size we will accept; this is important because we want to limit users from uploading gigabyte-sized files and filling up all of the server’s disk space. MAX_DIR_SIZE is the maximum size that we will allow our upload directory to grow to. This restriction is as important as the last because users can fill up our disks by posting lots of small files just as easily as posting large files. Finally, MAX_OPEN_TRIES is the number of times we try to generate a unique filename and open that file before we give up; we’ll see why this step is necessary in a moment.

First, we enable file uploads, then we set $CGI::POST_MAX to MAX_FILE_SIZE. Note $CGI::POST_MAX is actually the size of the entire content of the request, which includes the data for other form fields as well as overhead for the multipart/form-data encoding, so this value is actually a little larger than the maximum file size that the script will actually accept. For this form, the difference is minor, but if you add a file upload field to a complex form with multiple text fields, then you should keep this distinction in mind.

We then create a CGI object and check for errors. As we said earlier, errors with file uploads are much more common than with other forms of CGI input. Next we get the file’s upload name and the filename the user provided, reporting errors if either of these is missing. Note that a user may be rather upset to get a message saying that the filename is missing after uploading a large file via a modem. There is no way to interrupt that transfer, but in a production application, it might be more user-friendly to save the unnamed file temporarily, prompt the user for a filename, and then rename the file. Of course, you would then need periodically clean up temporary files that were abandoned.

We get a file handle, $fh, to the temporary file where CGI.pm has stored the input. We check whether our upload directory is full and report an error if this is the case. Again, this message is likely to create some unhappy users. In a production application you should add code to notify an administrator who can see why the upload directory is full and resolve the problem. See Chapter 9.

Next, we replace any characters in the filename the user supplied that may cause problems with an underscore and make sure the name doesn’t start with a period or a dash. The odd construct that reassigns the result of the regular expression to $filename untaints that variable. We’ll discuss tainting and why this is important in Chapter 8. We confirm again that $filename is not empty (which would happen if it had consisted of nothing but periods and/or dashes) and generate an error if this is the case.

We try to open a file with this name in our upload directory. If we fail, then we add a digit to $filename and try again. The regular expression allows us to keep the file extension the same: if there is already a report.txt file, then the next upload with that name will be named report1.txt, the next one report2.txt, etc. This continues until we exceed MAX_OPEN_TRIES . It is important that we create a limit to this loop because there may be a reason other than a non-unique name that prevents us from saving the file. If the disk is full or the system has too many open files, for example, we do not want to start looping endlessly. This error should also notify an administrator that something is wrong.

This script is written to handle any type of file upload, including binary files such as images or audio. By default, whenever Perl accesses a file handle on non-Unix systems (more specifically, systems that do not use as their end of line character), Perl translates the native operating system’s end of line characters, such as for Windows or for MacOS, to on input and back to the native characters on output. This works great for text files, but it can corrupt binary files. Thus, we enable binary mode with the binmode function in order to disable this translation. On systems, like Unix, where no end of line translation occurs, binmode has no effect.

Finally, we read from our temporary file handle and write to our output file and exit. We use the read function to read and write a chunk a data at a time. The size of this chunk is defined by our BUFFER_SIZE constant. In case you are wondering, CGI.pm will remove its temporary file automatically when our script exits (technically, when $q goes out of scope).

There is another way we could have moved the file to our uploads directory. We could use CGI.pm’s undocumented tmpFileName method to get the name of the temporary file containing the upload and then used Perl’s rename function to move the file. However, relying on undocumented code is dangerous, because it may not be compatible with future versions of CGI.pm. Thus, in our example we stick to the public API instead.

The dir_size subroutine calculates the size of a directory by summing the size of each of its files. The error subroutine prints a message telling the user why the transfer failed. In a production application, you probably want to provide links for the user to get help or to notify someone about problems.

CGI.pm has two methods for returning HTTP headers: header and redirect. They correspond to the two ways you can return data from CGI scripts: you can return a document, or you can redirect to another document.

print $q->header( "text/plain" );
print "This is just some boring text.
";

print $q->header( -type => "text/html", -status => "404 Not Found" );

print $q->header( -type => "text/html", -expires => "+30m" );

print $q->header( -type => "text/html", -target => "main_frame" );

print $q->redirect( "http://localhost/survey/thanks.html" );

print $q->header( -content_encoding  => "gzip" );

Content-encoding: gzip

Now let’s look at the methods that you can use to generate HTML. We’ll start by looking at the methods for starting and ending documents.

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<HTML><HEAD><TITLE>The Time</TITLE>
</HEAD><BODY BGCOLOR="#ffffff">

HTML elements can be generated by using the lowercase name of the element as a method, with the following exceptions: Accept, Delete, Link, Param, Select, Sub, and Tr. These methods have an initial cap to avoid conflicting with built-in Perl functions and other CGI.pm methods.

The following rules apply to basic HTML tags:

The syntax for generating form elements differs from other elements. These methods only take name-value pairs that correspond to the attributes. See Table 5.2.

The start_form and end_form elements generate the opening and closing form tags. start_form takes arguments for each of its attributes:

print $q->start_form( method => "get", action => "/cgi/myscript.cgi" );

Note that unlike a typical form tag, CGI.pm sets the request method to POST instead of GET by default (the reverse of the default for HTML forms). If you want to allow file uploads, use the start_multipart_form method instead of start_form, which sets enctype to “multipart/form-data”.

All of the remaining methods create form elements. They all take the -name and -default arguments. The -default value for an element is replaced by the corresponding value from param if that value exists. You can disable this and force the default to override a user’s parameters by passing the -override argument with a true value.

The -default option specifies the default value of the element for elements with single values:

print $q->textfield(
        -name    => "username",
        -default => "Anonymous"
      );

This yields:

<INPUT TYPE="text" NAME="username" VALUE="Anonymous">

By supplying an array with the -values argument, the checkbox_group and radio_group methods generate multiple checkboxes that share the same name. Likewise, passing an array reference with the -values argument to the scrolling_list and popup_menu functions generates both the <SELECT> and <OPTION> elements. For these elements, -default indicates the values that are checked or selected; you can pass -default a reference to an array for checkbox_group and scrolling_list for multiple defaults.

Each method accepts a -labels argument that takes a reference to a hash; this hash associates the value of each element to the label the browser displays to the user.

Here is how you can generate a group of radio buttons:

print $q->radio_group(
        -name    => "curtain",
        -values  => [ "A", "B", "C" ],
        -default => "B",
        -labels  => { A => "Curtain A", B => "Curtain B", C => "Curtain C" }
      );

This yields:

<INPUT TYPE="radio" NAME="look_behind" VALUE="A">Curtain A
<INPUT TYPE="radio" NAME="look_behind" VALUE="B" CHECKED>Curtain B
<INPUT TYPE="radio" NAME="look_behind" VALUE="C">Curtain C

For specifying any other attributes for form elements, like SIZE=4, pass them as additional arguments (e.g., size => 4).

The simplest solution for including HTML in the source code is the hardest to maintain. Many web developers start out writing CGI scripts that contain numerous print statements to return documents, even for large sections of static content—content that remains the same each time the CGI script is called.

Here is an example:

#!/usr/bin/perl -wT

use strict;

my $timestamp = localtime;

print "Content-type: text/html

";
print "<html>
";
print "<head>
";
print "<title>The Time</title>
";
print "</head>
";

print "<body bgcolor="#ffffff">
";
print "<h2>Current Time</h2>
";
print "<hr>
";
print "<p>The current time according to this system is: 
";
print "<b>$timestamp</b>
";
print "</p>
";
print "</body>
";
print "</html>
";

This is a pretty basic example, but you could imagine just how complicated this can get on a large web page with numerous graphics, nested tables, style declarations, etc. Not only is this difficult to read because of the extra noise that each print statement adds, but each double quote in the HTML must be escaped with a backslash. If you forget to do this even once, you will likely generate a syntax error. Making HTML edits to something that looks like this is much more work than it should be. You should definitely avoid this approach in your scripts.

As we have seen in earlier examples, Perl supports a feature called here documents that allows you to express a large block of content separately within your code. To create a here document, simply use << followed by the token that will be used to indicate the end of the here document. You can include the token in single or double quotes, and the content will be evaluated as if it were a string within those quotes. In other words, if you use single quotes, variables will not be interpreted. If you omit the quotes, it acts as though you had used double quotes.

Here is the previous example using a here document instead:

#!/usr/bin/perl -wT

use strict;

my $timestamp = localtime;

print <<END_OF_MESSAGE;
Content-type: text/html

<html>
  <head>
    <title>The Time</title>
  </head>
  
  <body bgcolor="#ffffff">
    <h2>Current Time</h2>
    <hr>
    <p>The current time according to this system is: 
    <b>$timestamp</b></p>
  </body>
</html>
END_OF_MESSAGE

This is much cleaner than using lots of print statements, and it allows us to indent the HTML content. The result is that this is much easier to read and to update. You could have accomplished something similar by using one print statement and putting all the content inside one pair of double quotes, but then you would have had to precede each double quote in the HTML with a backslash, and for complicated HTML documents this could get tedious.

Another solution is to use Perl’s qq// operator, but with a different delimiter, such as ~. You must find a delimiter that will not appear in the HTML, and remember that if your content includes JavaScript, it can include many characters that HTML might otherwise not. here documents are generally a safer solution.

One drawback to using here documents is that they do not easily indent, so they may look odd inside blocks of otherwise cleanly indented code. Tom Christiansen and Nathan Torkington address this issue in the Perl Cookbook (O’Reilly & Associates, Inc.). The following solutions are adapted from their discussion.

If you do not care about extra leading whitespace in your HTML output, you can simply indent everything. You can also indent the ending token if you use quotes and include the indent in the name (although this is more readable, it may be less maintainable because if the indentation changes, then you must adjust the name of the token to match):

#!/usr/bin/perl -wT

use strict;

my $timestamp = localtime;
display_document( $timestamp );

sub display_document {
    my $timestamp = shift;
    
    print <<"    END_OF_MESSAGE";
      Content-type: text/html
      
      <html>
        <head>
          <title>The Time</title>
        </head>
        
        <body bgcolor="#ffffff">
          <h2>Current Time</h2>
          <hr>
          <p>The current time according to this system is: 
          <b>$timestamp</b></p>
        </body>
      </html>
    END_OF_MESSAGE
}

One problem with indenting HTML here documents is that the extra indentation is sent to the client. You can solve this problem by creating a function that “unindents” your text. If you wish to remove all indentation, this is simple; if you want to maintain your HTML’s indentation, this is more complex. The challenge is determining the amount of indentation to remove: what portion belongs to the content and what part is incidental to your script? You could assume the first line contains the smallest indent, but this would not work if you were only printing the end of an HTML document, for example, when the last line would probably contain the smallest indent.

In the following code the unindent subroutine looks at all of the lines being printed, finds the smallest indent, and removes that amount from all of the lines:

sub unindent;

sub display_document {
    my $timestamp = shift;
    
    print unindent <<"    END_OF_MESSAGE";
      Content-type: text/html
      
      <html>
        <head>
          <title>The Time</title>
        </head>
        
        <body bgcolor="#ffffff">
          <h2>Current Time</h2>
          <hr>
          <p>The current time according to this system is: 
          <b>$timestamp</b></p>
        </body>
      </html>
    END_OF_MESSAGE
}

sub unindent {
    local $_ = shift;
    my( $indent ) = sort /^([ 	]*)S/gm;
    s/^$indent//gm;
    return $_;
}

Predeclaring the unindent function, as we do on the first line, allows us to omit parentheses when we use it. This solution, of course, increases the amount of work the server must do for each request, so it would not be appropriate on a heavily used server. Also keep in mind that each additional space increases the number of bytes you must transfer and the user must download, so you may actually want to strip all leading whitespace instead. After all, users probably care more about the page downloading faster than how it looks if they view the source code.

Overall, here documents are not a bad solution for large chunks of code, but they do not offer CGI.pm’s advantages, especially the ability to have your HTML code verified syntactically. It’s much harder to forget to close an HTML tag with CGI.pm than it is with a here document. Also, many times you must build HTML programmatically. For example, you may read records from a database and add a row to a table for each record. In these cases, when you are working with small chunks of HTML, CGI.pm is much easier to work with than here documents.

Using CGI.pm’s methods for outputting HTML generates strong reactions in developers. Some love it; others don’t. Don’t worry if it doesn’t match your needs, we will look at a whole class of alternatives in the next chapter.

The most common method that Perl developers use for handling errors is Perl’s built-in die function. Here is an example:

open FILE, $filename or die "Cannot open $filename: $!";

If Perl is unable to open the file specified by $filename, die will print an error message to STDERR and terminate the script. The open function, like most Perl commands that interact with the system, sets $! to the reason for the error if it fails.

Unfortunately, die is not always the best solution for handling errors in your CGI scripts. As you will recall from Chapter 3, output to STDERR is typically sent to the web server’s error log, triggering the web server to return a 500 Internal Server Error . This is certainly not a very user-friendly response.

You should determine a policy for handling errors on your site. You may decide that 500 Internal Server Error pages are acceptable for very uncommon system errors like the inability to read or write to files. However, you may decide that you wish to display a formatted HTML page instead with information for users such as alternative actions they can take or who to notify about the problem.

eval {
    dangerous_routine(  );
    1;
} or do {
    error( $q, $@ || "Unknown error" );
};

CGI::Carp is not part of the CGI.pm module, but it is also by Lincoln Stein, and it is distributed with CGI.pm (and thus included with the most recent versions of Perl). It does two things: it creates more informative entries in your error log, and it allows you to create a custom error page for fatal calls like die. Simply by using the module, it adds a timestamp and the name of the running CGI script to errors written to the error log by die , warn, carp, croak, and confess. The last three functions are provided by the Carp module (included with Perl) and are often used by module authors.

This still does not stop your web server from displaying 500 Internal Server Error responses for these calls, however. CGI::Carp is most useful when you ask it to trap fatal calls. You can have it display fatal error messages in the browser instead. This is especially helpful during development and debugging. To do this, simply pass the fatalsToBrowser parameter to it when you use the module:

use CGI::Carp qw( fatalsToBrowser );

In a production environment, you may not want users to view your full error information if they encounter an error. Fortunately, you can have CGI::Carp trap errors and display your own custom error message. To do this, you pass CGI::Carp::set_message a reference to a subroutine that takes a single argument and displays the content of a response.

use CGI::Carp qw( fatalsToBrowser );
BEGIN {
    sub carp_error {
        my $error_message = shift;
        my $q = new CGI;
          $q->start_html( "Error" ),
          $q->h1( "Error" ),
          $q->p( "Sorry, the following error has occurred: " );
          $q->p( $q->i( $error_message ) ),
          $q->end_html;
    }
    CGI::Carp::set_message( &carp_error );
}

We will see how to incorporate this into a more general solution later in Example 5.3.

Most of our examples up to now and throughout the book include subroutines or blocks of code for displaying errors. Here is an example:

sub error {
    my( $q, $error_message ) = shift;
    
    print $q->header( "text/html" ),
          $q->start_html( "Error" ),
          $q->h1( "Error" ),
          $q->p( "Sorry, the following error has occurred: " );
          $q->p( $q->i( $error_message ) ),
          $q->end_html;
    exit;
}

You can call this with a CGI object and a reason for the error. It will output an error page and then exit in order to stop executing your script. Note that we print the HTTP header here. One of the biggest challenges in creating a general solution for catching errors is knowing whether or not to print an HTTP header: if one has already been printed and you print another, it will appear at the top of your error page; if one has not been printed and you do not print one as part of the error message, you will trigger a 500 Internal Server Error instead.

Fortunately, CGI.pm has a feature that will track whether a header has been printed for you already. If you enable this feature, it will only output an HTTP header once per CGI object. Any future calls to header will silently do nothing. You can enable this feature in one of three ways:

Although adding subroutines to each of your CGI scripts is certainly an acceptable way to catch errors, it’s still not a very general solution. You will probably want to create your own error pages that are customized for your site. Once you start including complex HTML in your subroutines, it will quickly become too difficult to maintain them. If you build error subroutines that output error pages according to your site’s template, and then later someone decides they want to change the site’s look, you must go back and update all of your subroutines. Clearly, a much better option is to create a general error handler that all of your CGI scripts can access.

It is a good idea to create your own Perl module that’s specific to your site. If you host different sites, or have different applications within your site with different looks and feels, you may wish to create a module for each. Within this module, you can place subroutines that you find yourself using across many CGI scripts. These subroutines will vary depending on your site, but one should handle errors.

If you have not created your own Perl module before, don’t worry, it’s quite simple. Example 5.3 shows a very minimal module.

#!/usr/bin/perl -wT

package CGIBook::Error;

# Export the error subroutine
use Exporter;
@ISA = "Exporter";
@EXPORT = qw( error );

$VERSION = "0.01";

use strict;
use CGI;
use CGI::Carp qw( fatalsToBrowser );

BEGIN {
    sub carp_error {
        my $error_message = shift;
        my $q = new CGI;
        my $discard_this = $q->header( "text/html" );
        error( $q, $error_message );
    }
    CGI::Carp::set_message( &carp_error );
}

sub error {
    my( $q, $error_message ) = @_;
    
    print $q->header( "text/html" ),
          $q->start_html( "Error" ),
          $q->h1( "Error" ),
          $q->p( "Sorry, the following error has occurred: " ),
          $q->p( $q->i( $error_message ) ),
          $q->end_html;
    exit;
}

1;

The only difference between a Perl module and a standard Perl script is that you should save your file with a .pm extension, declare the name of module’s package with the package function (this should match the file’s name except without the .pm extension and substituting :: for /),^[7] and make sure that it returns a true value when evaluated (the reason for the 1; at the bottom).

It is standard practice to store the version of the module in $VERSION. For the sake of convenience, we also use the Exporter module to export the error subroutine. This allows us to refer to it in our scripts as error instead of CGIBook::Exporter::error. Refer to the Exporter manpage or a primary Perl text, such as Programming Perl, for details on using Exporter.

You have a couple options for saving this file. The simplest solution is to save it within the site_perl directory of your Perl libraries, such as /usr/lib/perl5/site_perl/5.005/CGIBook/Error.pm. The site_perl directory includes modules that are site-specific (i.e., not included in Perl’s standard distribution). The paths of your Perl libraries may differ; you can locate them on your system with the following command:

$ perl -e 'print map "$_
", @INC'

You probably want to create a subdirectory that is unique to your organization, as we did with CGIBook, to hold all the Perl modules you create.

You can use the module as follows:

#!/usr/bin/perl -wT

use strict;
use CGI;
use CGIBook::Error;

my $q = new CGI;

unless ( check_something_important(  ) ) {
    error( $q, "Something bad happened." );
}

If you do not have the permission to install the module in your Perl library directory, and if you cannot get your system administrator to do it, then you can place the module in another location, for example, /usr/local/apache/perl-lib/CGIBook/Error.pm. Then you must remember to include this directory in the list that Perl searches for modules. The simplest way to do this is with the lib pragma:

#!/usr/bin/perl -wT

use strict;
use lib "/usr/local/apache/perl-lib";

use CGI;
use CGIBook::Error;
.
.
.