Perl scripts generally start with three standard file handles predefined: STDIN, STDOUT, and STDERR. CGI Perl scripts are no different. These file handles have particular meaning within a CGI script, however.

The standard CGI environment variables listed in Table 3.1 should be available on any server supporting CGI. Nonetheless, if you loop through all the keys in %ENV, you will probably not see all the variables listed here. If you recall, some HTTP request headers are used only with certain requests. For example, the Content-length header is sent only with POST requests. The environment variables that map to these HTTP request headers will thus be missing when its corresponding header field is missing. In other words, $ENV{CONTENT_LENGTH} will only exist for POST requests.

Any HTTP headers that the web server does not recognize as standard headers, as well as a few other common headers, are also available to your script. The web server follows these rules for creating the name of the environment variable:

Table 3.2 provides a list of some of the more common of these environment variables.

A secure server typically adds many more environment variables for secure connections. Much of this information is based on X.509 and provides information about the server’s and possibly the browser’s certificates. Because you really won’t need to understand these details in order to write CGI scripts, we won’t get into X.509 or secure HTTP transactions in this book. For more information, refer to RFC 2511 or the public key infrastructure working group’s web site at http://www.imc.org/ietf-pkix/.

The names of the environment variables supplied to your script for secure connections vary by server. The HTTPS environment variable (see Table 3.3) is commonly supported, however, and useful to test whether your connection is secure; unfortunately its values vary between servers. Refer to your server’s documentation for more information or use Example 3.1 or Example 3.2 to generate data for your server.

Finally, the web server may provide additional environment variables beyond those mentioned in this section. Most web servers also allow the administrator to add environment variables via a configuration file. You might take advantage of this feature if you have several CGI scripts that all share common configuration information, such as the name of the database server to connect to. Having the variable defined once in the web server’s configuration file makes it easy to change later.

Because browsers and web servers may provide additional environment variables to your script, it’s often helpful to have a list of environment variables that is specific to your web server. Example 3.1 shows a short script that is easy to remember and type in when you find yourself working on a new system. It generates a handy list of environment variables specific to that web server. Remember that the browser may also affect this list. For example, HTTP_COOKIE will only appear if the browser supports cookies, if cookies have not been disabled, and if the browser had received a previous request from this web server to set a cookie.

#!/usr/bin/perl -wT
# Print a formatted list of all the environment variables

use strict;

print "Content-type: text/html

";

my $var_name;
foreach $var_name ( sort keys %ENV ) {
    print "<P><B>$var_name</B><BR>";
    print $ENV{$var_name};
}

This simply produces an alphabetic list of the environment variable names and their values, shown in Figure 3.2.

Because this is simply a quick-and-dirty script, we omitted some details that should be included in production CGI scripts, and which are included in the other examples. For example, we did not print a valid HTML document (it is missing the enclosing HTML, HEADER, and BODY tags). This should certainly be added if the script were to grow beyond a few lines or if you intended for people other than yourself to use it.

Example 3.2 shows a more elaborate version that displays all of the environment variables that CGI and your web server define, along with a brief explanation of the standard variables.

#!/usr/bin/perl -wT

use strict;

my %env_info = (
    SERVER_SOFTWARE     => "the server software",
    SERVER_NAME         => "the server hostname or IP address",
    GATEWAY_INTERFACE   => "the CGI specification revision",   
    SERVER_PROTOCOL     => "the server protocol name",
    SERVER_PORT         => "the port number for the server",
    REQUEST_METHOD      => "the HTTP request method",
    PATH_INFO           => "the extra path info",
    PATH_TRANSLATED     => "the extra path info translated",
    DOCUMENT_ROOT       => "the server document root directory",
    SCRIPT_NAME         => "the script name",
    QUERY_STRING        => "the query string",
    REMOTE_HOST         => "the hostname of the client",
    REMOTE_ADDR         => "the IP address of the client",
    AUTH_TYPE           => "the authentication method",
    REMOTE_USER         => "the authenticated username",
    REMOTE_IDENT        => "the remote user is (RFC 931): ",
    CONTENT_TYPE        => "the media type of the data",
    CONTENT_LENGTH      => "the length of the request body",
    HTTP_ACCEPT         => "the media types the client accepts",
    HTTP_USER_AGENT     => "the browser the client is using",
    HTTP_REFERER        => "the URL of the referring page",
    HTTP_COOKIE         => "the cookie(s) the client sent"
);

print "Content-type: text/html

";

print <<END_OF_HEADING;

<HTML>
<HEAD>
    <TITLE>A List of Environment Variables</TITLE>
</HEAD>

<BODY>
<H1>CGI Environment Variables</H1>

<TABLE BORDER=1>
  <TR>
    <TH>Variable Name</TH>
    <TH>Description</TH>
    <TH>Value</TH>
  </TR>
END_OF_HEADING

my $name;

# Add additional variables defined by web server or browser
foreach $name ( keys %ENV ) {
    $env_info{$name} = "an extra variable provided by this server"
        unless exists $env_info{$name};
}

foreach $name ( sort keys %env_info ) {
    my $info = $env_info{$name};
    my $value = $ENV{$name} || "<I>Not Defined</I>";
    print "<TR><TD><B>$name</B></TD><TD>$info</TD><TD>$value</TD></TR>
";
}

print "</TABLE>
";
print "</BODY></HTML>
";

Figure 3-2. Output of env.cgi

The %env_info hash contains the standard environment variable names and their descriptions. The while loop iterates over %ENV with the each command to add any additional environment variables defined by the current web server. Then the foreach loop iterates through the combined list and displays the name, description, and value of each environment variable. Figure 3.3 shows what the output will look in a browser window.

Figure 3-3. Output of env_info.cgi

This covers most of CGI input, but we have not discussed how to read the message body for POST requests. We will return to that topic when we discuss forms in the next chapter. Right now, let’s look at CGI output.

CGI scripts must output one of the following three headers:

Let’s review each of these options.

print "Content-type: text/html

";

print "Content-type: text/html

";

print "Content-type: text/html

";

Forwarding to another URL

Sometimes, it’s not necessary to build an HTML document with your CGI script. In fact, unless the output varies from one visit to another, it is a good idea to create a simple, static HTML page (in addition to the CGI script), and forward the user to that page by using the Location header. Why? Interface changes are far more common than program logic changes, and it is much easier to reformat an HTML page than to make changes to a CGI script. Plus, if you have multiple CGI scripts that return the same message, then having them all forward to a common document reduces the number of resources you need to maintain. Finally, you get better performance. Perl is fast, but your web server will always be faster. It’s a good idea to take advantage of any opportunity you have to shift work from your CGI scripts to your web server.

To forward a user to another URL, simply print the Location header with the URL to the new location:

print "Location: static_response.html

";

The URL may be absolute or relative. An absolute URL or a relative URL with a relative path is sent back to the browser, which then creates another request for the new URL. A relative URL with a full path produces an internal redirect. An internal redirect is handled by the web server without talking to the browser. It gets the contents of the new resource as if it had received a new request, but it then returns the content for the new resource as if it is the output of your CGI script. This avoids a network response and request; the only difference to users is a faster response. The URL displayed by their browser does not change for internal redirects; it continues to show the URL of the original CGI script. See Figure 3.4 for a visual display of server redirection.

Figure 3-4. Server redirection

When redirecting to absolute URLs, you may include a Content-type header and content body for the sake of older browsers, which may not forward automatically. Modern browsers will immediately fetch the new URL without displaying this content.

print <<END_OF_HTML;
Status: 503 Database Unavailable
Content-type: text/html

<HTML>
<HEAD><TITLE>503 Database Unavailable</TITLE></HEAD>
<BODY>
  <H1>Error</H1>
  <P>Sorry, the database is currently not available. Please
    try again later.</P>
</BODY>
</HTML>
END_OF_HTML

Thus far, all the CGI scripts that we’ve discussed simply return partial header information. We leave it up to the server to fill in the other headers and return the document to the browser. We don’t have to rely on the server though. We can also develop CGI scripts that generate a complete header.

CGI scripts that generate their own headers are called nph (non-parsed headers) scripts. The server must know in advance whether the particular CGI script intends to return a complete set of headers. Web servers handle this differently, but most recognize CGI scripts with a nph- prefix in their filename.

When sending complete headers, you must at least send the status line plus the Content-type and Server headers. You must print the entire status line; you should not print the Status header. As you will recall, the status line includes the protocol and version string (e.g., “HTTP/1.1”), but as you should recall, CGI provides this to you in the environment variable SERVER_PROTOCOL. Always use this variable in your CGI scripts, instead of hardcoding it, because the version in the SERVER_PROTOCOL may vary for older clients.

Example 3.3 provides a simple example that illustrates nph scripts.

#!/usr/bin/perl -wT

use strict;

print "$ENV{SERVER_PROTOCOL} 200 OK
";
print "Server: $ENV{SERVER_SOFTWARE}
";
print "Content-type: text/plain

";

print "OK, starting time consuming process ... 
";

# Tell Perl not to buffer our output
$| = 1;

for ( my $loop = 1; $loop <= 30; $loop++ ) {
    print "Iteration: $loop
";
    ## Perform some time consuming task here ##
    sleep 1;
}

print "All Done!
";

nph scripts were more common in the past, because versions of Apache prior to 1.3 buffered the output of standard CGI scripts (those generating partial headers) but did not buffer the output of nph scripts. By creating nph scripts, your output was sent immediately to the browser as it was generated. However Apache 1.3 no longer buffers CGI output, so this feature of nph scripts is no longer needed with Apache. Other web servers, such as iPlanet Enterprise Server 4, buffer both standard CGI as well as nph output. You can find out how your web server handles buffering by running Example 3.3.

Save the file as nph-count.cgi and access it from your browser; then save a copy as count.cgi and update it to output partial headers by commenting out the status line and the Server header:

# print "$ENV{SERVER_PROTOCOL} 200 OK
";
# print "Server: $ENV{SERVER_SOFTWARE}
";

Access this copy of the CGI script and compare the result. If your browser pauses for thirty seconds before displaying the page, then the server is buffering the output; if you see the lines displayed in real time, then it is not.

CGI scripts are not restricted to generating HTML. Example 3.4 produces an image after choosing an image format that the browser supports. Recall that the browser sends the Accept HTTP header listing the media types it supports. Actually, browsers generally specify only the newer media types they support and pass a wildcard to match everything else. In this example, we’ll send an image in the new PNG format if the browser specifies that it supports PNG, and a JPEG otherwise.

You may ask why we would want to do this. Well, JPEG images use a lossy form of compression. Although they are ideal for natural images like photographs, images with sharp lines and details (such as screenshots or text) can become blurred. PNG images, like GIF images, do not use lossy compression. They are typically larger than JPEG images (it depends on the image), but they provide sharp detail. And unlike GIFs, which are limited to 256 colors, PNGs can support millions of colors and even eight-bit transparency. So we will provide a high-color, high-detail PNG if possible, or a JPEG otherwise.

If a user calls this with http://localhost/cgi/image_fetch.cgi/new_screenshot.png, he or she will actually get new_screenshot.png or new_screenshot.jpeg depending on what the browser supports. This allows you to include a single link in your HTML pages that works for everyone. Example 3.4 shows the source to our CGI script.

#!/usr/bin/perl -wT

use strict;

my $image_type = $ENV{HTTP_ACCEPT} =~ m|image/png| ? "png" : "jpeg";
my( $basename ) = $ENV{PATH_INFO} =~ /^(w+)/;
my $image_path = "$ENV{DOCUMENT_ROOT}/images/$basename.$image_type";

unless ( $basename and -B $image_path and open IMAGE, $image_path ) {
    print "Location: /errors/not_found.html

";
    exit;
}

my $buffer;
print "Content-type: image/$image_type

";
binmode;

while ( read( IMAGE, $buffer, 16_384 ) ) {
    print $buffer;
}

We set $image_type to “png” or “jpeg” depending on whether the browser sent image/png as part of its Accept header. Then we set $basename to the first word of the additional path information, which is “new_screenshot” in our previous example. We only care about the base name because we add our own extension when we actually fetch the file.

Our images are in the images directory at the root of the web server’s document tree, so we build a path to the image and assign it to $image_path. Note that we build this path before we validate that the URL we received actually contains additional path information. If $ENV{PATH_INFO} is empty or starts with a nonalphanumeric character, then obviously this path is invalid. That’s okay though; we will validate this in the next step.

We delayed the validation so we can perform all of our tests at once. We test that the additional path information contains a name, that the full path to the file we constructed points to a binary file, and that we are able to open the file. If any of these tests fail, then we simply report that the file is not found. We do this by forwarding to a static page that contains our error message. Creating a single, static document for general errors like 404 Not Found is an easy way to produce error pages that are customized to match your site design and are easy to maintain.

If we opened the file successfully, we read and print the file in 16KB increments. Calling binmode is necessary for systems like Win32 or MacOS that do not use newlines as the end-of-line character; it doesn’t hurt on Unix systems.

In addition to domain-based security, most HTTP servers also support another method of security, known as user authentication. We discussed user authentication briefly in the last chapter. When configured for user authentication, specified files or directories within a given realm are set up to allow access only by certain users. A user attempting to open the URLs associated with these files is prompted for a name and password.

The username and password is checked by the server, and if legitimate, the user is allowed access. In addition to allowing the user access to the protected file, the server also maintains the user’s name and passes it to any subsequent CGI programs that are called. The server passes the username in the $ENV{REMOTE_USER} environment variable.

A CGI script can therefore use server authentication information to identify users. Here is a snippet of code that illustrates what you can do with the $ENV{REMOTE_USER} environment variable:

$remote_user = $ENV{REMOTE_USER};

if ( $remote_user eq "mary" ) {
    print "Welcome Mary, how is your company doing these days?
";
} elsif ( $remote_user eq "bob" ) {
    print "Hey Bob, how are you doing? I heard you were sick.
";
}

One of the great benefits of the Web is its flexibility. One person can create a page on their server and include links to others’ pages on other servers. These links can even include links to images on other servers. Unfortunately, if you have popular images, you may not appreciate this last feature. Say, for example, you are an artist and you display your images on your web site. You may not want other sites to include your artwork in their web pages simply by including image links pointing to your server. One solution, shown in Example 3.5, is to check the URL that referred the user to the image via the Referer HTTP header field.^[5]

#!/usr/bin/perl -wT

use strict;

# The directory where images are stored; this shouldn't be in the 
# server's doc tree so users can't browse images except via this script.
my $image_dir = "/usr/local/apache/data/images";

my $referer  = $ENV{HTTP_REFERER};
my $hostname = quotemeta( $ENV{HTTP_HOST} || $ENV{SERVER_NAME} );

if ( $referer and $referer !~ m|^http://$hostname/| ) {
    display_image( "copyright.gif" );
}
else {
    # Verify that the image name doesn't contain any unsafe characters.
    my( $image_file ) = $ENV{PATH_INFO} =~ /^([w+.]+)$/ or
        not_found(  );
    display_image( $image_file );
}
sub display_image {
    my $file = shift;
    my $full_path = "$image_dir/$file";
    
    # We'll simply report that the file isn't found if we can't open it.
    open IMAGE, $full_path or not_found(  );
    
    print "Pragma: no-cache
";
    print "Content-type: image/gif

";
    
    binmode;
    my $buffer = "";
    while ( read( IMAGE, $buffer, 16_384 ) ) {
        print $buffer;
    }
    close IMAGE;
}

sub not_found {
    print <<END_OF_ERROR;
Status: 404 Not Found
Content-type: text/html

<html>
<head>
  <title>File Not Found</title>
</head>

<body>
  <h1>File Not Found</h1>
  
  <p>Sorry, but you requested an image that could not be found. 
    Please check the URL you entered and try again.</p>
</body>
</html>
END_OF_ERROR
    
    exit;
}

This script displays an image with a copyright notice if the user came from a different web site. For the copyright notice, the script assumes that there is a file called copyright.gif in the same directory as the other images. Not all browsers implement the Referer HTTP header, and we don’t want visitors using these browsers to get the wrong image in error. So we only display the copyright image if the user both presents a Referer header and it is from a different server. Also, we have to be conscious of caching on the Web. Browsers might cache images, and they may be behind any number of proxy servers that also implement their own caches. Thus, we output an additional header to request that this message not be cached. This should avoid the user getting a cached copyright notice image when they visit the real site. If you are especially paranoid (and do not mind the extra traffic it causes), then you could also output a Pragma: no-cache header for the real images too.

If the image is not found, it sends a response with a 404 status. You may wonder why it would send an HTML message when it was likely the request was the result of an image tag and the browser is planning on displaying the response as an image in an HTML page. Actually, neither web servers nor CGI scripts have any way of determining the context of any request. Web servers always display 404 errors when they cannot locate a resource. In this case the browser will likely display an icon, such as a broken image, to indicate that there was an error. If the user chooses to view the image separately by directly referencing it, he or she will see the error message.

This solution should stop casual hijackers. It won’t stop thieves. It’s always possible for someone to visit your site, download your images, and put copies of them up on their own site.