This chapter covers in detail the most common issues found when running Apache, such as problems with file permissions and not being able to bind to a certain port, and how to fix them. It also explains several tools and resources available to isolate the cause of most problems.
We know there is nothing more frustrating than not been able to keep reading a technical book because you cannot get the software to work. We don’t want to be one of those books! Hence the reason to address this topic early on. Because of that, this chapter covers both basic and advanced topics, so feel free to skip the ones that do not apply to you if you are new to Apache.
The error log file keeps track of important events in the life of the server, including starts, restarts, warnings or errors related to the operation of the server, and forbidden or invalid requests. This is the first place to look when you are trying to solve a problem with the server.
On Unix systems, the error_log file is placed by default in the logs/ directory of your Apache installation. If you are using an installation of Apache that came with your distribution, this file may be in a different location, most commonly /var/log/httpd.
On Windows, the file is named error.log and placed under the logs directory as well.
Use the ErrorLog directive to specify the path to the error log file. Prefix the path to the program with a pipe to log errors to the standard input of another program. This common technique is described in detail in Chapter 3.
Note that the error file will not be created until the first time you start Apache!
ErrorLog syslog ErrorLog syslog:local7
On Unix systems, specify syslog as the argument to ErrorLog to instruct Apache to use the system log daemon to log Apache errors. This is shown in the example. You can optionally attach a facility (by default local7) as shown. A syslog facility is an information field that is associated with a syslog message to indicate the source of a log message. Facilities local0 to local10 are reserved for use by administrators and applications, such as Apache.
The error information provided by Apache can be categorized according to degrees of importance. Use the LogLevel directive, supplying one of the arguments shown in Table 2.1, to choose the messages that you want to receive. Only errors of that level of importance or higher will be logged.
Table 2.1. LogLevel Options As Described in the Apache Documentation
Setting | Description | Example |
|---|---|---|
Emergencies—system is unusable |
| |
Action must be taken immediately |
| |
Critical conditions |
| |
Error conditions |
| |
Warning conditions |
| |
Normal but significant condition |
| |
Informational |
| |
Debug-level messages |
|
The default error level of “warn” is appropriate for most Apache installations. If you are trying to troubleshoot a specific configuration, however, you can lower the level all the way to “debug” to get much more detailed logging information.
Use this command to test the Apache configuration file for common problems before you use it with a live server. Apache uses the same process to test your configuration each time you issue a restart command using apachectl. This guarantees that a running server will be able to restart successfully using the new configuration file.
$ telnet www.apache.org 80 Trying 192.87.106.226... Connected to ajax-1.apache.org (192.87.106.226). Escape character is '^]'. HEAD / HTTP/1.0 HTTP/1.1 200 OK Date: Sun, 04 Sep 2005 20:42:02 GMT Server: Apache/2.0.54 (Unix) mod_ssl/2.0.54 OpenSSL/0.9.7a DAV/2 SVN/1.2.0-dev Last-Modified: Sat, 03 Sep 2005 11:35:42 GMT ETag: "203a8-2de2-3ffdc7a6d3f80" Accept-Ranges: bytes Content-Length: 11746 Cache-Control: max-age=86400 Expires: Mon, 05 Sep 2005 20:42:02 GMT Connection: close Content-Type: text/html; charset=ISO-8859-1 Connection closed by foreign host.
Because HTTP is a simple text-based protocol, you can use a telnet client, a program that allows you to connect directly to a server and port you specify, to test for the presence of a working Apache server from the command line. If you receive no response to a remote telnet request and are positive that your network is properly configured, Apache is not listening on the address and port in question. This can be useful for troubleshooting in environments where a web browser is not available, as can be the case when accessing a server remotely over SSH. For example, if you can access Apache in a remote machine from the localhost address using telnet, but not remotely using a browser, it may indicate firewall problems or an incorrect setting of the Apache Listen directive.
Connect via telnet to www.apache.org (or your favorite website) at port 80 and type
HEAD / HTTP/1.0
or
GET / HTTP/1.0
Press the Enter key twice. You will get a response similar to the example.
If you have the lynx command-line browser installed in your Unix system, you can get a similar result by issuing the command
lynx –head –dump http://www.apache.org
Chapter 7 covers mod_ssl and explains a similar way to connect to an SSL-enabled web server using the openssl command-line tool.
ps -aux | grep httpd 25297 ? S 0:00 /usr/local/www/bin/ httpd -k start 15874 ? S 0:06 /usr/local/www/bin /httpd -k start 14441 ? S 0:02 /usr/local/www/bin /httpd -k start ... /usr/sbin/lsof | grep httpd |grep IPv httpd 14441 nobody 3u IPv4 136524 TCP www.example.com:http (LISTEN) httpd 25297 root 3u IPv4 136524 TCP www.example.com:http (LISTEN) httpd 30277 nobody 3u IPv4 136524 TCP www.example.com:http (LISTEN) ... netstat -ltnp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.1.151:80 0.0.0.0: * LISTEN 25297/httpd tcp 0 0 0.0.0.0:22 0.0.0.0: * LISTEN 1038/sshd
Sometimes, you may not be able to connect to the server and are therefore unsure of whether the server is running or there is a network problem. In Unix systems, you can use a number of command-line tools to help you find out. The example shows some of them.
The ps tool shows whether or not the httpd process is running on the system.
The netstat and lsof tools show the port and address on which the Apache server is listening.
In Windows systems, you can use the Windows task manager (invoked by pressing Ctrl–Alt–Del) to see whether the Apache.exe process is running. Alternatively, you can use the Apache monitor tray application included with recent distributions to check the status of Apache.
Sometimes it is necessary or convenient to signal the server directly using the kill command-line utility instead of the apachectl wrapper script. To do this, first find the process id of the running Apache server using ps or lsof as shown . Then, end the process with the kill command-line tool, supplying the signal to be sent as the first argument and the Apache server’s process id (25297 in this example) as the second argument. Use HUP as the signal to stop the server or SIGHUP as the signal to restart the server. You can also replace the signal with its numerical equivalent, as shown in the example. Read the kill manual page for details.
In Linux, you can send a signal to all processes named httpd with the killall command. For example, you can kill all httpd processes using
# killall –HUP httpd
You need to be careful, because if you are running several Apache instances, this command will take them all down!
Note that you need to have the appropriate permissions for these commands to work. In nearly all cases, you must either be the superuser or the owner of the Apache process in order to end or restart it.
In Windows systems, you can force Apache to stop by using the Windows Task Manager and pressing the End Task button.
There are number of Apache modules that can help you when troubleshooting or debugging an Apache setup or a web application.
mod_loopback, a Web client debugging tool, simply echoes back to the browser everything received concerning an HTTP request, including POST or PUT data.
http://www.snert.com/Software/mod_loopback/index.shtml
mod_tee and mod_trace_output are third-party modules that store the content as it is being served. They can be found at these URLs:
http://apache.webthing.com/mod_tee/
http://trace-output.sourceforge.net/
mod_logio, distributed with Apache 2, dumps all data received or returned by the server into the error log for debugging purposes.
All of these modules have an effect on performance, but can be very useful when debugging, for example, header or cookie-related issues.
This section explains a number of problems that may prevent Apache from starting and the error that you will receive for each.
Invalid command ‘PiidFile’, perhaps misspelled or defined by a module not included in the server configuration
A syntax error indicates that you have misspelled a directive (in this case, PidFile) in the configuration file or that you have included one or more directives used by a module that has not been added to the server. Check the syntax of the configuration file indicated in the error message. See Chapter 1 for details on using an <ifModule> directive to conditionally exclude directives so that the configuration file can still be processed when a module is not available.
"Address already in use: make_sock: could not bind to port"
An address already in use error means that another program is using the port Apache is trying to bind to. To solve the problem, either stop the program that is using that port before starting Apache, or edit the httpd.conf configuration file and change the port on which Apache will listen for requests by adjusting the values given after the Listen and Port directives.
In most cases an address already in use error happens because another Apache server is already running on your system or, in the case of Windows, an Internet Information Server or Microsoft Personal Web Server instance is running on the port Apache has been configured to use. Other popular programs, such as Skype, are also known to use port 80 on certain occasions.
A could not bind to port error indicates that you do not have the necessary permissions to request that Apache bind to the port specified in the Apache configuration file. On Unix, only privileged users can bind to ports between 1 and 1024. To solve this problem, log in as root or issue the su command and try to start the server again. If you do not have root access, edit your httpd.conf file and change the port that Apache uses to a number greater than 1024.
"module xxx is not compatible with this version of Apache"
A module not compatible error occurs when Apache tries to load a module that was compiled for a newer (or older) Apache server than the one currently installed. If you have the source code for the module, you may be able to recompile it using your current Apache installation, as shown in Chapter 1. If you do not have source code for or are unable to recompile a module whose functionality is essential to you, upgrade (or downgrade) your Apache server to a version compatible with the module.
Several Apache directives, including ServerName and Listen, accept hostnames as arguments. However, if Apache is not able to resolve a supplied hostname to an IP address at startup time using the Domain Name Service (DNS) or your system’s host list, you will receive the cannot determine hostname error. To solve the problem, verify your DNS and /etc/hosts settings and the spellings of hostnames supplied in httpd.conf. Whenever possible, use IP addresses for directives such as Listen and <VirtualHost> instead of hostnames.
(13)Permission denied: httpd: could not open error log file /usr/local/apache/logs/error_log.
A permission denied error indicates that you do not have sufficient permissions either to read the Apache configuration file(s) or to write to the Apache log files.
This problem often happens when Apache is launched by a different user than the one who built and installed it. Either start Apache as superuser or as the user that installed it, or, if you have sufficient permission to do so, use chmod to change ownership of the file named in the error message.
If your web browser returns 403 Forbidden/Access Denied when you attempt to load a web page through your Apache server, it means that the requested URL is subject to access restrictions that were not met by your request. To solve this problem, change the permissions of the web content or files that you have requested, and ensure that all directories leading to the document in question are both read- and execute-accessible to the owner of the Apache process. In Unix systems, you can use the chmod utility to set those permissions.
A “Client denied by server configuration” statement in the error log indicates that the denial results from access control directives (such as Allow and Deny) in the <Directory> or <Location> sections for that URL in your Apache configuration files.
A “Directory index forbidden by rule” statement in the error log indicates that you have attempted to view a directory in which no index file can be found. For details on directory indexing and index files, read about the Indexes option of the Options directive, covered in Chapter 6.
Options ExecCGI is off in this directory
If when attempting to execute a CGI-script you see “Options ExecCGI is off in this directory,” it means that you have not marked the CGI as executable in the Apache configuration file or that CGI scripts cannot be run from the directory in question. Read about the ScriptAlias or Options directive for more information.
Internal server errors are errors that prevent Apache from fulfilling a request.
A segmentation fault occurs when the Apache server attempts to access memory areas belonging to other system processes or a malformed or illegal instruction is encountered by the system in the Apache process. This is caused either by a bug, usually in poorly written or experimental libraries or modules, or by hardware faults, usually in the system memory, chipset, bus, or processor.
[error] [client 192.168.200.3] Premature end of script headers: /usr/local/apache/cgi-bin/test-cgi
A premature end of script headers error is caused by incomplete CGI script execution. Make sure the CGI program has executable permissions and that the interpreter in the first line of the script points to the correct program. For example, you will receive this error if your script begins with #!/usr/local/bin/perl on its first line when in reality your Perl interpreter is located at /usr/bin/perl.
“Premature end of headers” errors are generally due to abnormal program termination before the script has returned any data. Program failures can be caused by a variety of additional reasons, including errors in your code or missing libraries to which the program is linked. In some cases, the operating system or Apache might terminate the process if its resource usage (memory, CPU time) exceeds a certain limit, as explained in Chapter 9.
[error] [client 192.168.200.3] malformed header from script. Bad header=xxx: /usr/local/apache/cgi-bin/ example.cgi
A malformed header from script error occurs when headers are not in the appropriate format (usually because of a programming error). The Apache server expects the response from the script to start with zero or more headers, followed by an empty line.
RewriteLog /usr/local/apache/logs/rewrite_log RewriteLogLevel warn SSLLog /usr/local/apache/logs/ssl_log SSLLogLevel warn ScriptLog logs/cgi_log
A number of modules, including the Apache 1.3 SSL module, mod_rewrite, and mod_cgi provide their own directive for logging module-specific data to a separate file.
UseCanonicalName offIf your Apache server becomes unreachable whenever the server issues a redirect, it may be because the canonical name of your host is inaccessible from outside your network or incorrect.
For example, a ServerName set to localhost, 127.0.0.1, or a private address will be inaccessible if the server redirects the user to a URL based on these values.
To solve this problem, provide a valid ServerName or set UseCanonicalName to “off” so that self-referential URLs are constructed with the hostname provided by the client. This is a common issue with machines behind a reverse proxy, which is discussed in Chapter 10.
This section summarizes some of the most common issues found when troubleshooting an Apache problem.
If you cannot successfully start the server, check the error log for information on why the failure occurred.
If another server is already running at that address, choose a different address/port combination for your server.
If you do not have permissions to bind to the requested port, start Apache as the superuser (root) so that you have access to bind to privileged ports.
If Apache is unable to open the configuration or log files, ensure that the files are owned by the same user that installed Apache and that the user in question has permission to write to them.
If you are trying to access a page in the server and it does not display, to solve the problem you must first try to isolate whether it is caused by the server, network, or browser.
First, ensure that Apache is running using ps, netstat, or the Task Manager (in Windows). It may be that the server is not running at all.
Then ensure that you can connect to Apache from the local machine. To do this, use telnet to connect directly to the server and issue a sample request.
Next, ensure that Apache running on the correct address/port combination. If you can access the server locally, but not remotely, Apache is likely listening on a local address or port that is not accessible remotely. Use netstat or lsof to determine on which addresses Apache is listening and ensure that they are correct.
Ensure that your firewall or router is correctly configured. If Apache is listening to the correct address but is inaccessible outside your network, network traffic to your Apache server may be blocked. Use the traceroute utility (tracert on Windows) to test for network connectivity between the hosts in question. Many operating systems prevent access from the outside by default except on a few selected ports. If you are running Apache on a nonstandard port, you may be blocked. How to fix this varies from distribution to distribution. For example, you can use the system-config-securitylevel tool on Fedora systems and the Windows Firewall tool in the Windows Control Panel.
Finally, if you are using Secure Sockets Layer (SSL) to access the server (explained in Chapter 7) and you are connecting using older browser versions or running unusual configurations, check the error log for problems related to SSL data encryption.
If you can access the server, but you get a “Document not found” error, ensure that the document does indeed exist in the file system.
Then, ensure that the request reached the server by checking the access_log file for request(s) from the host in question. If you have multiple Apache instances running simultaneously, it may be that you are connecting to the wrong server.
Next, ensure that your Alias directives point to the right location—that is, to the location your target documents are located. Make sure that you did not misspell or accidentally delete the target directory.
Finally, check for incorrect redirects, including “trailing slashes” and the ServerName issues described earlier in this chapter.
If the document exists but you are told that you have been forbidden from accessing it, check for a number of common errors.
Ensure that the owner of the Apache process has permission to read the file.
Ensure that the owner of the Apache process has read and list permissions for all directories in the path leading to the file.
Check to see whether you are trying to access a directory without an index file when directory listings are forbidden in the Apache configuration file.
Verify that the request meets all of the requirements outlined by the access control directives in the Apache configuration file.
If you are trying to access a CGI-script, ensure that it has been given read and execute permissions.
If you get an “Internal server error” in your browser when you try to load a page from the web server, check the Apache error_log to try to find the cause. Ask yourself the following questions:
Are you trying to access a CGI program? Does it have the right read and execute permissions? Is the path to the interpreter in the first line of the script correct? Is it marked as a CGI script by a ScriptAlias directive or similar?
This chapter has discussed only the most common problems faced by Apache users. If you encounter a problem not covered in this chapter, the first step toward resolving it is to check the error logs for details. Increase the Apache server’s LogLevel, if necessary, to find hints as to what the problem may be. Search the Apache documentation, mailing lists, and bug database. Finally, post your question to the Apache Users mailing list at the following address, taking care to follow the posting guidelines when doing so: Do your homework first, then provide enough information for others to be able to help.