20 Using Network and Protocol Functions
IN THIS CHAPTER, WE LOOK AT THE NETWORK-ORIENTED functions in PHP that enable your scripts to interact with the rest of the Internet. A world of resources is available out there, and a wide variety of protocols is available for using them.
Key topics covered in this chapter include
Using data from other websites
Using network lookup functions
Examining Available Protocols
Protocols are the rules of communication for a given situation. For example, you know the protocol when meeting another person: You say hello, shake hands, communicate for a while, and then say goodbye. Different situations require different protocols. Also, people from other cultures may expect different protocols, which may make interaction difficult. Computer networking protocols are similar.
Like human protocols, different computer protocols are used for different situations and applications. For instance, you use the Hypertext Transfer Protocol (HTTP) when you request and receive web pages—your computer requests a document (such as an HTML or PHP file) from a web server, and that server responds by sending the document to your computer. You probably also have used the File Transfer Protocol (FTP) for transferring files between machines on a network. Many others are available.
Most protocols and other Internet standards are described in documents called Requests For Comments (RFCs). These protocols are defined by the Internet Engineering Task Force (IETF). The RFCs are widely available on the Internet. The base source is the RFC Editor at http://www.rfc-editor.org/.
If you have problems when working with a given protocol, the documents that define them are the authoritative sources and are often useful for troubleshooting your code. They are, however, very detailed and often run to hundreds of pages.
Some examples of well-known RFCs are RFC2616, which describes the HTTP/1.1 protocol, and RFC822, which describes the format of Internet email messages.
In this chapter, we look at aspects of PHP that use some of these protocols. Specifically, we discuss sending mail with SMTP, reading mail with POP3 and IMAP4, connecting to other web servers via HTTP, and transferring files with FTP.
Sending and Reading Email
The main way to send mail in PHP is to use the simple mail() function. We discussed the use of this function in Chapter 4, “String Manipulation and Regular Expressions,” so we won’t visit it again here. This function uses the Simple Mail Transfer Protocol (SMTP) to send mail.
You can use a variety of freely available classes to add to the functionality of mail(). In Chapter 30, “Building a Mailing List Manager,” you use an add-on class to send HTML attachments with a piece of mail. SMTP is only for sending mail. The Internet Message Access Protocol (IMAP4), described in RFC2060, and Post Office Protocol (POP3), described in RFC1939 or STD0053, are used to read mail from a mail server. These protocols cannot send mail.
IMAP4 is used to read and manipulate mail messages stored on a server and is more sophisticated than POP3, which is generally used simply to download mail messages to a client and delete them from the server.
PHP comes with an IMAP4 library. It can also be used to make POP3 and Network News Transfer Protocol (NNTP) as well as IMAP4 connections.
We look extensively at the use of the IMAP4 library in the project described in Chapter 29, “Building a Web-Based Email Service.”
Using Data from Other Websites
One of the great things you can do with the Web is use, modify, and embed existing services and information into your own pages. PHP makes this very easy. Let’s look at an example to illustrate this use.
Imagine that the company you work for wants the company’s stock quote displayed on its home page. This information is available on some stock exchange site somewhere, but how do you get at it?
Start by finding an original source URL for the information. When you know the URL, every time someone goes to your home page, you can open a connection to that URL, retrieve the page, and pull out the information you require.
As an example, we put together a script that retrieves and reformats a stock quote from the AMEX website. For the purpose of the example, we retrieved the current stock price of Amazon.com. (The information you want to include on your page might differ, but the principles are the same.)
Our example script consumes a web service provided by another site to display data provided by them on our site. The script is shown in Listing 20.1.
Listing 20.1 lookup.php—Script Retrieves a Stock Quote from the NASDAQ for the Stock with the Ticker Symbol Listed in $symbol
<html>
<head>
<title>Stock Quote From NASDAQ</title>
</head>
<body>
<?php
//choose stock to look at
$symbol = 'AMZN';
echo '<h1>Stock quote for ' . $symbol . '</h1>';
$url = 'http://finance.yahoo.com/d/quotes.csv' .
'?s=' . $symbol . '&e=.csv&f=sl1d1t1c1ohgv';
if (!($contents = file_get_contents($url))) {
die('Failure to open ' . $url);
}
// extract relavant data
list($symbol, $quote, $date, $time) = explode(',', $contents);
$date = trim($date, '"'),
$time = trim($time, '"'),
echo '<p>' . $symbol . ' was last sold at: ' . $quote . '</p>';
echo '<p>Quote current as of ' . $date . ' at ' . $time . '</p>';
// acknowledge source
echo '<p>This information retrieved from <br /><a href="' . $url .
'">' . $url . '</a>.</p>';
?>
</body>
</html>
The output from one sample run of Listing 20.1 is shown in Figure 20.1.
Figure 20.1 The lookup.php script uses a regular expression to pull out the stock quote from information retrieved from the stock exchange.
The script itself is reasonably straightforward; in fact, it doesn’t use any functions you haven’t seen before, just new applications of those functions.
You might recall that when we discussed reading from files in Chapter 2, “Storing and Retrieving Data,” we mentioned that you could use the file functions to read from an URL. That’s what we have done in this case. The call to file_get_contents():
if (!($contents = file_get_contents($url))) {
returns the entire text of the web page at that URL stored in $contents.
The file functions can do a lot in PHP. The example here simply loads a web page via HTTP, but you could interact with other servers via HTTPS, FTP, or other protocols in exactly the same way. For some tasks, you might need to take a more specialized approach. Some FTP functionality is available in the specific FTP functions, and not available via fopen() and other file functions. There is an example using the FTP functions later in this chapter. For some HTTP or HTTPS tasks, you may need to use the cURL library. With cURL, you can log in to a website and mimic a user’s progress through a few pages.
Having obtained the text of the page from file_get_contents(), you can then use the list() function to find the part of the page that you want:
list($symbol, $quote, $date, $time) = explode(',', $contents);
$date = trim($date, '"'),
$time = trim($time, '"'),
echo '<p>' . $symbol . ' was last sold at: ' . $quote . '</p>';
echo '<p>Quote current as of ' . $date . ' at ' . $time . '</p>';
That’s it!
You can use this approach for a variety of purposes. Another good example is retrieving local weather information and embedding it in your page.
The best use of this approach is to combine information from different sources to add some value. You can see one good example of this approach in Philip Greenspun’s infamous script that produces the Bill Gates Wealth Clock at http://philip.greenspun.com/ WealthClock.
This page takes information from two sources. It obtains the current U.S. population from the U.S. Census Bureau’s site. It also looks up the current value of a Microsoft share and combines these two pieces of information, adds a healthy dose of the author’s opinion, and produces new information—an estimate of Bill Gates’ current worth.
One side note: If you’re using an outside information source such as this for a commercial purpose, it’s a good idea to check with the source or take legal advice first. You might need to consider intellectual property issues in some cases.
If you’re building a script like this, you might want to pass through some data. For example, if you’re connecting to an outside URL, you might like to pass some parameters that would normally be typed in by the user. If you’re doing this, it’s a good idea to use the urlencode() function. This function takes a string and converts it to the proper format for an URL; for example, transforming spaces into plus signs. You can call it like this:
$encodedparameter = urlencode($parameter);
One problem with this overall approach is that the site you’re getting the information from may change its data format, which will stop your script from working.
Using Network Lookup Functions
PHP offers a set of “lookup” functions that can be used to check information about hostnames, IP addresses, and mail exchanges. For example, if you were setting up a directory site such as Yahoo! when new URLs were submitted, you might like to automatically check that the host of an URL and the contact information for that site are valid. This way, you can save some overhead further down the track when a reviewer comes to look at a site and finds that it doesn’t exist or that the email address isn’t valid.
Listing 20.2 shows the HTML for a submission form for a directory like this.
Listing 20.2 directory_submit.html—HTML for the Submission Form
<html>
<head>
<title>Submit your site</title>
</head>
<body>
<h1>Submit site</h1>
<form method=post action="directory_submit.php">
URL: <input type=text name="url" size=30 value="http://"><br />
Email contact: <input type=text name="email" size=23><br />
<input type="submit" name="Submit site">
</form>
</body>
</html>
This is a simple form; the rendered version, with some sample data entered, is shown in Figure 20.2.
Figure 20.2 Directory submissions typically require your URL and some contact details so directory administrators can notify you when your site is added to the directory.
When the submit button is clicked, you want to first check that the URL is hosted on a real machine, and, second, that the host part of the email address is also on a real machine. We wrote a script to check these things, and the output is shown in Figure 20.3.
Figure 20.3 This version of the script displays the results of checking the hostnames for the URL and email address; a production version might not display these results, but it is interesting to see the information returned from the checks.
The script that performs these checks uses two functions from the PHP network functions suite: gethostbyname() and dns_get_mx(). The full script is shown in Listing 20.3.
Listing 20.3 directory_submit.php—Script to Verify URL and Email Address
<html>
<head>
<title>Site submission results</title>
</head>
<body>
<h1>Site submission results</h1>
<?php
// Extract form fields
$url = $_REQUEST['url'];
$email = $_REQUEST['email'];
// Check the URL
$url = parse_url($url);
$host = $url['host'];
if(!($ip = gethostbyname($host)) )
{
echo 'Host for URL does not have valid IP';
exit;
}
echo "Host is at IP $ip <br>";
// Check the email address
$email = explode('@', $email);
$emailhost = $email[1];
// note that the dns_get_mx() function is *not implemented* in
// Windows versions of PHP
if (!dns_get_mx($emailhost, $mxhostsarr))
{
echo 'Email address is not at valid host';
exit;
}
echo 'Email is delivered via: ';
foreach ($mxhostsarr as $mx)
echo "$mx ";
// If reached here, all ok
echo '<br>All submitted details are ok.<br>';
echo 'Thank you for submitting your site.<br>'
.'It will be visited by one of our staff members soon.'
// In real case, add to db of waiting sites…
?>
</body>
</html>
Let’s go through the interesting parts of this script.
First, you take the URL and apply the parse_url() function to it. This function returns an associative array of the different parts of an URL. The available pieces of information are the scheme, user, pass, host, port, path, query, and fragment. Typically, you don’t need all these pieces, but here’s an example of how they make up an URL.
Consider the following URL at http://nobody:[email protected]:80/script.php?variable=value#anchor.
The values of each of the parts of the array are
scheme: http
user: nobody
pass: secret
host: example.com
port: 80
script: /path.php
query: variable=value
fragment: anchor
In the directory_submit.php script, you want only the host information, so you pull it out of the array as follows:
$url = parse_url($url);
$host = $url['host'];
After you’ve done this, you can get the IP address of that host, if it is in the domain name service (DNS). You can do this by using the gethostbyname() function, which returns the IP if there is one or false if not:
$ip = gethostbyname($host);
You can also go the other way by using the gethostbyaddr() function, which takes an IP as a parameter and returns the hostname. If you call these functions in succession, you might well end up with a different hostname from the one you began with. This can mean that a site is using a virtual hosting service where one physical machine and IP address host more than one domain name.
If the URL is valid, you then go on to check the email address. First, you split it into username and hostname with a call to explode():
$email = explode('@', $email);
$emailhost = $email[1];
When you have the host part of the address, you can check to see whether there is a place for that mail to go by using the dns_get_mx() function:
dns_get_mx($emailhost, $mxhostsarr);
This function returns the set of Mail Exchange (MX) records for an address in the array you supply at $mxhostsarr.
An MX record is stored at the DNS and is looked up like a hostname. The machine listed in the MX record isn’t necessarily the machine where the email will eventually end up. Instead, it’s a machine that knows where to route that email. (There can be more than one; hence, this function returns an array rather than a hostname string.) If you don’t have an MX record in the DNS, there’s nowhere for the mail to go.
Note that the dns_get_mx() function is not implemented in Windows versions of PHP. If you are using Windows, you should look into the PEAR::Net_DNS package, which will work for you (http://pear.php.net/package/NET_DNS).
If all these checks are okay, you can put this form data in a database for later review by a staff member.
In addition to the functions you just used, you can use the more generic function checkdnsrr(), which takes a hostname and returns true if any record of it appears in the DNS.
Backing Up or Mirroring a File
File Transfer Protocol, or FTP, is used to transfer files between hosts on a network. Using PHP, you can use fopen() and the various file functions with FTP as you can with HTTP connections, to connect to and transfer files to and from an FTP server. However, a set of FTP-specific functions also comes with the standard PHP install.
These functions are not built into the standard install by default. To use them under Unix, you need to run the PHP configure program with the --enable-ftp option and then rerun make. If you are using the standard Windows install, FTP functions are enabled automatically.
(For more details on configuring PHP, see Appendix A, “Installing PHP and MySQL.”)
Using FTP to Back Up or Mirror a File
The FTP functions are useful for moving and copying files from and to other hosts. One common use you might make of this capability is to back up your website or mirror files at another location. Let’s look at a simple example using the FTP functions to mirror a file. This script is shown in Listing 20.4.
Listing 20.4 ftp_mirror.php—Script to Download New Versions of a File from an FTP Server
<html>
<head>
<title>Mirror update</title>
</head>
<body>
<h1>Mirror update</h1>
<?php
// set up variables - change these to suit application
$host = 'ftp.cs.rmit.edu.au';
$user = 'anonymous';
$password = '[email protected]';
$remotefile = '/pub/tsg/teraterm/ttssh14.zip';
$localfile = '/tmp/writable/ttssh14.zip';
// connect to host
$conn = ftp_connect($host);
if (!$conn)
{
echo 'Error: Could not connect to ftp server<br />';
exit;
}
echo "Connected to $host.<br />";
// log in to host
$result = @ftp_login($conn, $user, $pass);
if (!$result)
{
echo "ERROR: Could not log on as $user<br />";
ftp_quit($conn);
exit;
}
echo "Logged in as $user<br />";
// check file times to see if an update is required
echo 'Checking file time…<br />';
if (file_exists($localfile))
{
$localtime = filemtime($localfile);
echo 'Local file last updated ';
echo date('G:i j-M-Y', $localtime);
echo '<br />';
}
else
$localtime=0;
$remotetime = ftp_mdtm($conn, $remotefile);
if (!($remotetime >= 0))
{
// This doesn’t mean the file’s not there, server may not support mod time
echo 'Can't access remote file time.<br />';
$remotetime=$localtime+1; // make sure of an update
}
else
{
echo 'Remote file last updated ';
echo date('G:i j-M-Y', $remotetime);
echo '<br />';
}
if (!($remotetime > $localtime))
{
echo 'Local copy is up to date.<br />';
exit;
}
// download file
echo 'Getting file from server…<br />';
$fp = fopen ($localfile, 'w'),
if (!$success = ftp_fget($conn, $fp, $remotefile, FTP_BINARY))
{
echo 'Error: Could not download file';
ftp_quit($conn);
exit;
}
fclose($fp);
echo 'File downloaded successfully';
// close connection to host
ftp_quit($conn);
?>
</body>
</html>
The output from running this script on one occasion is shown in Figure 20.4.
Figure 20.4 The FTP mirroring script checks whether the local version of a file is up to date and downloads a new version if not.
The ftp_mirror.php script is quite generic.You can see that it begins by setting up some variables:
$host = 'ftp.cs.rmit.edu.au';
$user = 'anonymous';
$password = '[email protected]';
$remotefile = '/pub/tsg/teraterm/ttssh14.zip';
$localfile = '/tmp/writable/ttssh14.zip';
The $host variable should contain the name of the FTP server you want to connect to, and the $user and $password correspond to the username and password you would like to log in with.
Many FTP sites support what is called anonymous login—that is, a freely available username that anybody can use to connect. No password is required, but it is a common courtesy to supply your email address as a password so that the system’s administrators can see where their users are coming from. We followed this convention here.
The $remotefile variable contains the path to the file you would like to download. In this case, you are downloading and mirroring a local copy of Tera Term SSH, an SSH client for Windows. (SSH stands for secure shell. This is an encrypted form of Telnet.)
The $localfile variable contains the path to the location where you are going to store the downloaded file on your machine. In this case, you create a directory called /tmp/writable with permissions set up so that PHP can write a file there. Regardless of your operating system, you need to create this directory for the script to work. If your operating system has strong permissions, you will need to make sure that they allow your script to write. You should be able to change these variables to adapt this script for your purposes.
The basic steps you follow in this script are the same as if you wanted to manually transfer the file via FTP from a command-line interface:
1. Connect to the remote FTP server.
2. Log in (either as a user or anonymous).
3. Check whether the remote file has been updated.
4. If it has, download it.
5. Close the FTP connection.
Let’s consider each of these steps in turn.
Connecting to the Remote FTP Server
The first step is equivalent to typing
ftp hostname
at a command prompt on either a Windows or Unix platform. You accomplish this step in PHP with the following code:
$conn = ftp_connect($host);
if (!$conn)
{
echo 'Error: Could not connect to ftp server<br />';
exit;
}
echo "Connected to $host.<br />";
The function call here is to ftp_connect(). This function takes a hostname as a parameter and returns either a handle to a connection or false if a connection could not be established. The function can also take the port number on the host to connect to as an optional second parameter. (We did not use this parameter here.) If you don’t specify a port number, it will default to port 21, the default for FTP.
Logging In to the FTP Server
The next step is to log in as a particular user with a particular password. You can achieve this by using the ftp_login() function:
$result = @ftp_login($conn, $user, $pass);
if (!$result)
{
echo "ERROR: Could not log on as $user<br />";
ftp_quit($conn);
exit;
}
echo "Logged in as $user<br />";
The function takes three parameters: an FTP connection (obtained from ftp_connect()), a username, and a password. It returns true if the user can be logged in and false if she can’t. Notice that we put an @ symbol at the start of the line to suppress errors. We did this because, if the user cannot be logged in, a PHP warning appears in the browser window. You can catch the error as we have done here by testing $result and supplying your own, more user-friendly error message.
Notice that if the login attempt fails, you actually close the FTP connection by using ftp_quit(). We discuss this function more later.
Checking File Update Times
Given that you are updating a local copy of a file, checking whether the file needs updating first is sensible because you don’t want to have to redownload a file, particularly a large one, if it’s up to date. This way, you can avoid unnecessary network traffic. Let’s look at the code that checks file update times.
File times are the reason that you use the FTP functions rather than a much simpler call to a file function. The file functions can easily read and, in some cases, write files over network interfaces, but most of the status functions such as filemtime() do not work remotely.
To begin deciding whether you need to download a file, you check that you have a local copy of the file by using the file_exists() function. If you don’t, obviously you need to download the file. If it does exist, you get the last modified time of the file by using the filemtime() function and store it in the $localtime variable. If it doesn’t exist, you set the $localtime variable to 0 so that it will be “older” than any possible remote file modification time:
echo 'Checking file time…<br />';
if (file_exists($localfile))
{
$localtime = filemtime($localfile);
echo 'Local file last updated ';
echo date('G:i j-M-Y', $localtime);
echo '<br />';
}
else
$localtime=0;
(You can read more about the file_exists() and filemtime() functions in Chapters 2, “Storing and Retrieving Data” and 19, “Interacting with the File System and the Server,” respectively.)
After you have sorted out the local time, you need to get the modification time of the remote file. You can get this time by using the ftp_mdtm() function:
$remotetime = ftp_mdtm($conn, $remotefile);
This function takes two parameters—the FTP connection handle and the path to the remote file—and returns either the Unix timestamp of the time the file was last modified or –1 if there is an error of some kind. Not all FTP servers support this feature, so you might not get a useful result from the function. In this case, you can choose to artificially set the $remotetime variable to be “newer” than the $localtime variable by adding 1 to it. This way, you ensure that an attempt is made to download the file:
if (!($remotetime >= 0))
{
// This doesn’t mean the file’s not there, server may not support mod time
echo 'Can’t access remote file time.<br>';
$remotetime=$localtime+1; // make sure of an update
}
else
{
echo 'Remote file last updated ';
echo date('G:i j-M-Y', $remotetime);
echo '<br>';
}
When you have both times, you can compare them to see whether you need to download the file:
if (!($remotetime > $localtime))
{
echo 'Local copy is up to date.<br>';
exit;
}
Downloading the File
At this stage, you try to download the file from the server:
echo 'Getting file from server…<br>';
$fp = fopen ($localfile, 'w'),
if (!$success = ftp_fget($conn, $fp, $remotefile, FTP_BINARY))
{
echo 'Error: Could not download file';
fclose($fp);
ftp_quit($conn);
exit;
}
fclose($fp);
echo 'File downloaded successfully';
You open a local file by using fopen(), as you learned previously. After you have done this, you call the function ftp_fget(), which attempts to download the file and store it in a local file. This function takes four parameters. The first three are straightforward: the FTP connection, the local file handle, and the path to the remote file. The fourth parameter is the FTP mode.
The two modes for an FTP transfer are ASCII and binary. The ASCII mode is used for transferring text files (that is, files that consist solely of ASCII characters), and the binary mode is used for transferring everything else. Binary mode transfers a file unmodified, whereas ASCII mode translates carriage returns and line feeds into the appropriate characters for your system (
for Unix,
for Windows, and
for Macintosh).
PHP’s FTP library comes with two predefined constants, FTP_ASCII and FTP_BINARY, that represent these two modes. You need to decide which mode fits your file type and pass the corresponding constant to ftp_fget() as the fourth parameter. In this case, you are transferring a ZIP file, so you use the FTP_BINARY mode.
The ftp_fget() function returns true if all goes well or false if an error is encountered. You store the result in $success and let the user know how it went.
After the download has been attempted, you close the local file by using the fclose() function.
As an alternative to ftp_fget(), you could use ftp_get(), which has the following prototype:
int ftp_get (int ftp_connection, string localfile_path,
string remotefile_path, int mode)
This function works in much the same way as ftp_fget() but does not require the local file to be open. You pass it the system filename of the local file you would like to write to rather than a file handle.
Note that there is no equivalent to the FTP command mget, which can be used to download multiple files at a time. You must instead make multiple calls to ftp_fget() or ftp_get().
Closing the Connection
After you have finished with the FTP connection, you should close it using the ftp_quit() function:
ftp_quit($conn);
You should pass this function the handle for the FTP connection.
Uploading Files
If you want to go the other way—that is, copy files from your server to a remote machine—you can use two functions that are basically the opposite of ftp_fget() and ftp_get(). These functions are called ftp_fput() and ftp_put(). They have the following prototypes:
int ftp_fput (int ftp_connection, string remotefile_path, int fp, int mode)
int ftp_put (int ftp_connection, string remotefile_path,
string localfile_path, int mode)
The parameters are the same as for the _get equivalents.
Avoiding Timeouts
One problem you might face when transferring files via FTP is exceeding the maximum execution time. You know when this happens because PHP gives you an error message. This error is especially likely to occur if your server is running over a slow or congested network, or if you are downloading a large file, such as a movie clip.
The default value of the maximum execution time for all PHP scripts is defined in the php.ini file. By default, it’s set to 30 seconds. This is designed to catch scripts that are running out of control. However, when you are transferring files via FTP, if your link to the rest of the world is slow or if the file is large, the file transfer could well take longer than this.
Fortunately, you can modify the maximum execution time for a particular script by using the set_time_limit() function. Calling this function resets the maximum number of seconds the script is allowed to run, starting from the time the function is called. For example, if you call
set_time_limit(90);
the script will be able to run for another 90 seconds from the time the function is called.
Using Other FTP Functions
A number of other FTP functions are useful in PHP. The function ftp_size() can tell you the size of a file on a remote server. It has the following prototype:
int ftp_size(int ftp_connection, string remotefile_path)
This function returns the size of the remote file in bytes or –1 if an error occurs. It is not supported by all FTP servers.
One handy use of ftp_size() is to work out the maximum execution time to set for a particular transfer. Given the file size and speed of your connection, you can take a guess as to how long the transfer ought to take and use the set_time_limit() function accordingly.
You can get and display a list of files in a directory on a remote FTP server by using the following code:
$listing = ftp_nlist($conn, dirname($remotefile));
foreach ($listing as $filename)
echo "$filename <br>";
This code uses the ftp_nlist() function to get a list of names of files in a particular directory.
In terms of other FTP functions, almost anything that you can do from an FTP command line, you can do with the FTP functions. You can find the specific functions corresponding to each FTP command in the PHP online manual at http://us2.php.net/manual/en/ref.ftp.php.
The exception is mget (multiple get), but you can use ftp_nlist() to get a list of files and then fetch them as required.
Further Reading
We covered a lot of ground in this chapter, and as you might expect, a lot of material is out there on these topics. For information on the individual protocols and how they work, you can consult the RFCs at http://www.rfc-editor.org/.
You might also find some of the protocol information at the World Wide Web Consortium interesting; go to http://www.w3.org/Protocols/.
You can also try consulting a book on TCP/IP such as Computer Networks by Andrew Tanenbaum.
Next
We are now ready to move on to Chapter 21, “Managing the Date and Time,” and look at PHP’s libraries of date and calendar functions. There, you see how to convert from user-entered formats to PHP formats to MySQL formats, and back again.