Chapter 11. Logging

Good logging is important for real-time incident detection and after-the-fact auditing. By religiously watching your log file, you will often get warnings that an outage is about to occur or that an attacker is analyzing your network for vulnerabilities. This allows you to take action to correct or prevent the problem. Log files also provide an audit trail for determining what went wrong or what an attacker accomplished.

Logging in General

Logging is a balance between collecting as much useful information as possible and not collecting so much information that it overwhelms you. An administrator overwhelmed by log files will ignore them until after an incident occurs. This negates the first benefit of log files—early warning. Many network administrators keep the default logging setting on routers and never take advantage of all of the additional information that can be logged. The rest of this chapter covers the logging capabilities of Cisco routers and discusses how to avoid being overwhelmed by your log files.

To achieve maximum benefit from log files, you must monitor them regularly. On systems of medium importance, log review can be done daily with the results emailed to the administrator. On highly secure systems, log analysis is often done in real time with the results sent to a pager.

Numerous commercial and public domain software packages can help you analyze your log files. Use them. These packages automate the process of analyzing log files by filtering out uninteresting messages and highlighting the dangerous ones. Such programs are the only way an administrator can keep up with log files to use them for incident detection.

One of the most useful ways to set up your log-watching programs is to have them:

  1. Highlight everything you know is dangerous.

  2. Ignore everything you know is safe.

  3. Print out everything else.

Before you run the log checker the first time, go through your current logs and configure the software to highlight or ignore each different entry. Each day, run the log-check software, taking note of any highlighted (dangerous) messages. Next, for each message the software didn’t highlight or ignore, configure the software to do one or the other. After about two weeks of doing this daily, you will have a custom configuration that will automatically inform you of dangerous log messages while stripping out all extraneous ones. If all your routers log to a central syslog server, monitoring your logs becomes even easier, since everything can be monitored from one location and you need only a single log-checking configuration.

Router Logging

Cisco routers can log information in six ways:

Console logging

Console log messages are configured to be displayed on the console port only. You must be connected to the console port to see these messages.

Buffered logging

Buffered logging keeps the log messages in RAM on the router. A logging buffer must be configured on the router, and this buffer is circular, meaning that when it fills up, the oldest log message is deleted to make room for the new message.

Terminal logging

Using the terminal monitor command, routers can be configured to send log messages to the VTY terminals.

syslog

Cisco routers can be configured to send their log messages to one or more external syslog servers.

SNMP traps

If enabled, SNMP can be configured to send traps to an external SNMP server to log specific conditions.

AAA accounting

If you are using AAA, you can configure the router to log network connections and even command execution to the Network Access Server (NAS).

Log messages are given a priority from 0 to 7. The lower the number, the more critical the message. These levels are defined as shown in Table 11-1.

Table 11-1. Log severity levels

Level

Title

Description

0

Emergencies

System is unusable.

1

Alerts

Immediate action is needed.

2

Critical

A critical condition has occurred.

3

Errors

An error condition has occurred.

4

Warning

A warning message.

5

Notifications

Normal but significant events.

6

Informational

Information messages.

7

Debugging

Debugging messages.

These levels are a hierarchy in which messages of your chosen level and above are logged and printed. Choosing level 7 (debugging) will print out all messages from level 7 to 0, while choosing level 2 (critical) will only print out messages of levels 2 to 0.

Timestamps

The importance of accurate time was addressed in Chapter 10. It is also important to make sure that all log messages include detailed time information. Without this information, it becomes impossible to precisely correlate various log files to determine when an incident happened. To configure Cisco routers to include detailed time information in all log messages, use the service timestamps log datetime command. Use the following options to make sure as much information as possible is captured:

msec

This option includes milliseconds in each log timestamp. Without this option, log messages are timestamped only to the nearest second.

localtime

This option configures the routers to use the local time to timestamp each log message. Doing so is generally advisable to make the logs easier for a human to interpret, but may be left out when routers across different time zones are logging to a central syslog server.

show-timezone

This option configures the router to indicate the time zone when timestamping each message. This step makes comparing log messages from various sources much easier.

To enable this detailed timestamping:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#service timestamps log datetime msec localtime show-timezone
RouterOne(config)#^Z

Console Logging

To see console messages, you must be logged into the console. The console is configured by default to level 5 (notifications), meaning that it only displays messages of notification level or above.

Changing the console logging level

To change the logging level of the console, use the logging console command, followed by the number or name of the new severity level. To see all logging messages, set the console to level 7 (debug) by:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging console 7
RouterOne(config)#^Z

or

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging console debugging
RouterOne(config)#^Z

Disabling console logging

As you begin logging more information, you may have trouble using the console as the display is covered up by numerous log messages. You can eliminate some of these messages by changing to a higher severity level using the commands described or you can disable logging to the console altogether. Since each console log message requires a CPU interrupt to be delivered, disabling logging in the console port can increase a router’s performance. Disabling console logging is done by:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#no logging console
RouterOne(config)#^Z

Buffered Logging

Buffered logging keeps copies of log messages in the router’s RAM. This makes it very easy for an administrator logged into the router to display current log messages. The RAM buffer is circular. The buffer is a fixed size, and as new messages are added to the buffer, the older messages are deleted. This keeps the most recent messages in the buffer without filling up the router’s memory.

The size of each router’s logging buffer should be configured with regard to the amount of RAM on the router. These buffers should be large enough to keep useful log entries, but small enough not to use up the router’s RAM and interfere with performance. A size of 16,000 or 32,000 bytes is usually sufficient and doesnt overwhelm the router. Play with the values if you find that you need log messages kept longer.

To enable buffered logging:

  1. Make sure logging is turned on with the logging on command.

  2. Configure the buffer with the logging buffered command.

  3. Configure the logging buffer severity level with the logging buffered command.

The following example configures the router to use a 32,000-byte logging buffer and to log messages of level 6 (informational) and above:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging on
RouterOne(config)#logging buffered 32000
RouterOne(config)#logging buffered informational
RouterOne(config)#^Z

Terminal Monitor

While logging is enabled to the console by default, log messages are not normally sent to the VTYs. If you are logged in through the network on a VTY and want to see log messages on your terminal, use the terminal monitor command:

  1. Use the logging monitor command to configure the severity of messages sent to the VTY terminal.

  2. At the enable prompt of the terminal, type terminal monitor.

The following example configures the terminal to receive level 3 (error) messages and above:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging monitor error
RouterOne(config)#^Z
RouterOne#terminal monitor

Note that the terminal monitor command was run not from configuration mode, but rather from privileged or enable mode.

Warning

Turning on terminal monitor, especially at the debug level, can overwhelm your connection, so be careful. If you find your terminal session scrolling with so many log messages that you are overwhelmed, use the terminal no monitor command to stop the terminal from receiving the log messages. Even if you can’t see what you are typing, try typing terminal no monitor and pressing Enter; the router will still take the command and stop the log messages to your terminal.

syslog

syslog logging is the most important non-AAA method of logging. All previous logging methods print the log message to a terminal or keep the log in memory. Neither of these records the log messages long term. Long-term storage is critical for keeping audit trails and is required for analyzing logs after messages have left the terminal screen or have buffered out of memory.

Almost all Unix servers come with syslog servers configured by default. Almost all security-conscious organization have one or two central syslog servers in which all logging information is sent and recorded. These messages are sent across the network to the syslog server where the server records the message to the hard drive. Centralizing your log files in this method makes correlating log information from different routers and systems much easier.

Remember to configure your router to send all log messages to a syslog server so that messages are permanently recorded and not lost when the router runs out of memory or messages scroll off the terminal screen.

syslog facilities

syslog servers rely on the severity levels defined earlier and on another variable called the facility. syslog facilities separate different services so that log messages can be separated into different log files. This separation makes reviewing and auditing log files much easier. Table 11-2 lists some standard syslog facilities.

Table 11-2. syslog facilities

syslog facility

Description

User

Log messages generated by user processes

Kern

Log messages generated by the kernel

Mail

Log messages regarding the mail system

Daemon

Log messages generated by system daemons

Auth

Log messages regarding the authorization system

LPR

Log messages regarding the printing system

News

Log messages regarding Usenet news processes

UUCP

Reserved for the UUCP system

Cron

Log messages regarding the cron system

Local0

Reserved for your use

Local1

Reserved for your use

Local2

Reserved for your use

Local3

Reserved for your use

Local4

Reserved for your use

Local5

Reserved for your use

Local6

Reserved for your use

Local7

Reserved for your use (default Cisco log level)

Cisco routers are usually configured to use one of the reserved facilities local0 through local7.

Note

The syslog server must be configured to receive and record the log messages to a file. On almost all Unix systems, the syslog configuration file is /etc/syslog.conf. The standard format for syslog.conf files is (with tabs, not spaces):

Facility.Severity                                   logfile

If you are unfamiliar with syslog, the easiest way to get a router logging to the syslog server is to use a facility that is already configured. However, this facility mixes your Cisco log messages with server log messages. The recommended way to get the router logged into the server is to configure a unique facility, such as local6 or local7, to send messages to a different file. For example, to configure the syslog server to send all log messages it receives that have a facility of local6 and a severity of informational to the file /var/log/cisco, your /etc/syslog.conf file might contain:

local6.info                                 /var/log/cisco

You would then configure each router to use facility local6 when sending messages to the syslog server.

Configuring syslog logging

To configure syslog logging on your router:

  1. Configure the syslog facility using the logging facility command.

  2. Configure the syslog severity level with the logging trap command.

  3. Configure the syslog server to which log messages will be sent with the logging command.

To configure your router to send log messages to the server 13.18.10.4 using facility local6 and severity informational:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging facility local6
RouterOne(config)#logging trap informational
RouterOne(config)#logging 13.18.10.4
RouterOne(config)#^Z

You can specify additional syslog servers with additional logging <ip-address> commands and specify that log messages will be sent to each server for redundancy.

Warning

By default, many syslog servers accept messages from any IP address. While this won’t let an attacker modify log messages already recorded, it does allow an attacker to send bogus log messages that get mixed in with your good messages. It can also allow an attacker to use up all of the disk space on your syslog server so that valid log messages can’t be recorded because of lack of space.

It is important to make sure your syslog server does not accept syslog messages from systems external to your network. This can be accomplished by using ACLs at your network boundaries to deny all incoming syslog packets (UDP port 514) and/or by configuring the syslog server to accept log messages only from your internal network.

syslog sequence numbers

Cisco has recently added a feature that stamp each syslog message with a sequence number. These numbers increase by one for each syslog message sent. These sequence number can provide a way for an administrator to determine whether the log files have been tampered with. A log file that is missing a set of sequence numbers indicates that the log files are either incomplete or have been tampered with. This example enables syslog sequence numbers:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#service sequence-numbers  
RouterOne(config)#^Z

Sequence numbers do little if intruders accesses the syslog server—they can simply resequence the messages in the log file. However, sequence numbers can be useful in determining whether bogus messages are being forwarded to a syslog server. A log file that contains log messages without sequence numbers or with duplicate sequence numbers indicates that someone is sending bogus message to your syslog server.

Throttling syslog messages

IOS Version 12.1(3)T introduced the ability to limit the number of syslog messages sent to and from the router in a specific time frame. Some logging, such as ACL violation logging, if not implemented correctly, can cause the router to overwhelm the syslog server with thousands of messages a second. You can limit the number of messages sent with the logging rate-limit command. This example limits message output to 10 messages a second:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging rate-limit all 10
RouterOne(config)#^Z

You can append the except keyword followed by a severity level to rate-limit only low-priority messages.

For example, to avoid rate-limiting messages of severity error or higher, enter:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#logging rate-limit all 10 except error
RouterOne(config)#^Z

SNMP Traps

If you run SNMP (see Chapter 8), you can use SNMP traps to log additional information. Traps are packets sent to an SNMP server when specific events occur. Such events include high temperatures, configuration changes, and down interfaces. If you are familiar with SNMP and want to enable SNMP traps on your router, you must:

  1. Use the snmp-server host command to configure which SNMP server will receive the traps.

  2. Use the snmp-server enable traps command to enable SNMP traps.

The following example configures the router to send traps to the SNMP server 13.145.6.5:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#snmp-server host 13.145.6.5 public
RouterOne(config)#snmp-server enable traps 
RouterOne(config)#^Z

This example configures the router to send all possible traps to the SNMP server. You can supply additional arguments to the snmp-server enable traps command to limit the traps sent. See Cisco’s documents on SNMP traps for more information.

ACL Violation Logging

Aside from system logging, ACL violation logging is an important area to configure. By logging ACL violations, you can record any time your ACLs block access. This is extremely useful for determining when attackers are trying to spoof, log in, or access your router from external or untrusted systems.

ACL violation logging can be enabled by simply adding the keyword log or log-input to the end of your ACL statements. Be selective about what you choose to log. Adding the log keyword to the wrong filters can end up logging every packet in and out of your networks.

The log keyword provides information on the type, date, and time of the ACL violation and is the only option for standard ACLs. The log-input keyword is available on extended ACLs and logs additional information about the input interface and source MAC address.

Warning

The log keyword works only for standard access lists. Both the log and log-input keywords can be used for extended and named ACLs.

Antispoofing Violations

The following examples configure your ACL antispoofing filters to log both inbound and outbound spoofing attempts. Assuming our internal network was 130.18.0.0/16, an ingress filter that logs all spoofed packets looks like:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#access-list 115 deny ip 130.18.0.0 0.0.255.255 any log-input
RouterOne(config)#access-list 115 permit ip any any
RouterOne(config)#^Z

This filter would be placed inbound on each interface that borders an external network. Assuming this was Serial 0/0 on RouterOne, enter:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#int Serial 0/0
RouterOne(config-if)#ip access-group 115 in
RouterOne(config-if)#^Z

An egress filter that logs all attempts at sending spoofed packets out of the network would look like:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#access-list 116 permit ip 130.18.0.0 0.0.255.255 any
RouterOne(config)#access-list 116 deny ip any any log-input
RouterOne(config)#^Z

And would be applied to all outgoing interfaces:

RouterOne#config terminal         
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#int Serial 0/0
RouterOne(config-if)#ip access-group 116 out
RouterOne(config-if)#^Z

VTY Access Logging

Logging router login attempts through VTYs is very important. It can give you an early indication that someone is trying to access your router without authorization. Assuming you want only the IP 130.18.5.6 to be able to access the router through VTYs, your ACL would look like:

RouterOne#config terminal         
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#access-list 117 permit ip host 130.18.5.6 any
RouterOne(config)#access-list 117 deny ip any any log-input
RouterOne(config)#^Z

You would apply it the VTY lines 0 through 4 with:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#line vty 0 4
RouterOne(config-line)#access-class 117 in
RouterOne(config-line)#^Z

Now someone from an unauthorized IP trying to access a router VTY will generate a log message. The following message is an sample log entry of an unauthorized person trying to telnet to the router:

Oct 13 21:10:44.185 EDT: %SEC-6-IPACCESSLOGP: list 120 denied 
tcp 19.8.59.41(63104) -> 0.0.0.0(23), 1 packet

Many sites also choose to log all permitted VTY access attempts in addition to blocked access ones. Since all VTY access is through TCP, you can use the ACL keyword established to log successful access attempts without overwhelming your router or syslog server. This can be done effectively if you:

  • Pass, without logging, all established TCP connections from permitted IPs.

  • Pass and log the first packet for all TCP sessions from permitted IPs.

  • Deny and log all other packets.

This example logs both access attempts from approved IPs and access attempts from unapproved IP addresses:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#access-list 117  permit tcp host 130.218.5.6  any established 
RouterOne(config)#access-list 117  permit tcp host 130.218.5.6  any log-input
RouterOne(config)#access-list 117  deny ip any any log-input                
RouterOne(config)#^Z

Other Services

Other protocols and services such as HTTP, SNMP, ICMP, and route filtering can be logged in the same way. Just add the keyword log or log-input to the deny statements of the ACLs for each service you want to log.

AAA Accounting

AAA accounting allows you to log additional information that cannot be obtained from the previous logging methods. Using Cisco’s TACACS+, AAA accounting can log every command executed on the router to the Network Access Server (NAS).

AAA Accounting Methods

There are five methods of AAA accounting—EXEC, System, Command, Connection, and Network. Here is a short description of each method:

EXEC accounting

EXEC accounting records information about each EXEC, or shell, session on the router. It records information such as username, date, time, and IP of the system connecting to the router.

System accounting

System accounting logs information about system-level events, such as reboots or when accounting is turned on or off.

Command accounting

Command accounting logs information about the commands typed at the EXEC shell prompt. It records information (including what command was executed, who executed the command, the privilege level, and the date and time). Please note that older versions of Cisco’s implementation of RADIUS do not support Command accounting, so you will need to use TACACS+ or upgrade.

Connection accounting

Connection accounting logs information about outgoing connections made from the router. These connections include telnet, rlogin, tn3270, and LAT.

Network accounting

Network accounting logs information about PPP, SLIP, and ARAP sessions.

AAA Accounting Types

To log the largest amount of useful information, you should turn on accounting for all five methods. Each method supports three types of logging:

start-stop

The start-stop keyword configures the router to generate a log message when a service starts and stops. For example, using the start-stop keyword for EXEC, accounting generates a log message when a user first accesses the router and another log messages when the user disconnects from the router.

stop-only

The stop-only keyword configures the router to generate log messages only when a service ends. With EXEC accounting, this keyword generates log messages only when a user disconnects from the router.

wait-start

The wait-start keyword delays the start of the requested service until confirmation is received from the NAS server that a log message has been received. This is typically reserved for higher-security installations that want to guarantee that each connection and command that run on a system is recorded. If the NAS server doesn’t indicate that it successfully received the log message, the router will not start the requested service or command.

AAA Accounting Configurations

Recommended initial settings for each method are shown in Table 11-3.

Table 11-3. Recommended AAA method types

Method

Recommended starting type

Explanation

EXEC

start-stop

This method makes determining exactly when someone accessed the router and exactly when they disconnected much easier.

System

stop-only

System messages are usually atomic events that don’t need the start and stop times recorded. The stop-only keyword is usually sufficient for System accounting.

Command

stop-only

Commands are usually run in such short time periods that start and stop times are of limited use and can double the amount of log messages you have to analyze.

Connection

start-stop

Logging the start and stop times of outgoing connections from the router makes analyzing log files easier.

Network

start-stop

Again, logging start and stop times of these network connections makes log file analysis less of a chore and more intuitive.

The rest of this section provides examples on configuring each logging method.

Accounting with TACACS+

If you have AAA configured to use TACACS+, you can enable EXEC, System, Connection, and Network accounting to your NAS server with:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#aaa accounting exec default start-stop  group tacacs+
RouterOne(config)#aaa accounting system default stop-only  group tacacs+
RouterOne(config)#aaa accounting connection default start-stop  group tacacs+
RouterOne(config)#aaa accounting network default start-stop  group tacacs+
RouterOne(config)#^Z

To configure Command accounting, you also need to supply the privilege level you want logged. Unless you have changed the default privilege levels, privilege levels 1 and 15 should capture everything. If you have custom privilege-level configurations, make sure you add commands to log all levels you are interested in:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#aaa accounting commands 1  default stop-only  group tacacs+
RouterOne(config)#aaa accounting commands 15  default stop-only  group tacacs+
RouterOne(config)#^Z

Accounting with RADIUS

If you have AAA configured to use RADIUS, you can enable EXEC, System, Connection, and Network accounting to your NAS server with:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#aaa accounting exec default start-stop  group radius
RouterOne(config)#aaa accounting system default stop-only  group radius
RouterOne(config)#aaa accounting connection default start-stop  group radius
RouterOne(config)#aaa accounting network default start-stop  group radius
RouterOne(config)#^Z

You also need to enable Command accounting for all privilege levels you are interested in. Older versions of Cisco’s IOS don’t support Command accounting through RADIUS. If you determine that your router doesn’t support RADIUS command logging, then either migrate to TACACS+ or upgrade your IOS:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#aaa accounting commands 1  default stop-only  group radius
RouterOne(config)#aaa accounting commands 15  default stop-only  group radius
RouterOne(config)#^Z

AAA authentication failure logging

AAA also needs to be configured to log authentication failures. The following example will generate an AAA log message each time someone attempts to access the router but fails:

RouterOne#config terminal
Enter configuration commands, one per line.  End with CNTL/Z.
RouterOne(config)#aaa accounting send stop-record authentication failure 
RouterOne(config)#^Z

Logging Checklist

This checklist summarizes the important security information presented in this chapter. A complete security checklist is provided in Appendix A.

  • Actively monitor all logs for indications of attacks, misconfigurations, and failures.

  • Configure logging timestamps to include milliseconds using the service timestamp log datetime msec localtime command.

  • Enable RAM buffer logging with the logging buffered command. The default and recommended level is 6 (informational).

  • Enable logging sequence numbers with the service sequence-numbers command.

  • Configure routers to send log messages to a syslog server to preserve the messages:

    • Make sure that sites requiring higher levels of security and auditability send router log messages to multiple syslog servers for redundancy.

    • Filter out syslog messages from external systems through ACLs at your network’s border or with the syslog server itself.

  • Configure key ACLs to record access violations. Recommended ACL logging includes:

    • Antispoofing violations

    • VTY access attempts

    • HTTP access attempts

    • SNMP access attempts

    • Route filter violations

    • ICMP violations

    • Any other important filters

  • In environments requiring additional security, use AAA and enable AAA accounting:

    • Configure EXEC, System, Connection, and Network accounting to record information on system events and user sessions.

    • Configure AAA accounting to record authentication failures.

    • If a record of each command executed on the router is required, configure command accounting.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.225.55.151