Chapter 10. Troubleshooting Remote Access VPNs

This chapter covers the following topics:

• Troubleshooting Clientless SSLVPNs on the ASA: This section explores the key steps involved in troubleshooting a Cisco clientless SSLVPN deployment on the ASA, including SSL errors and conditional debugging techniques.

• Troubleshooting AnyConnect SSLVPNs on the ASA: This section steps through various Cisco commands for troubleshooting an AnyConnect SSL deployment on an ASA.

• Troubleshooting AnyConnect IKEv2 VPNs on the ASA: This section examines the unique issues related to an IKEv2 VPN deployment and how the troubleshooting process occurs.

• Troubleshooting AnyConnect IKEv2 VPNs on Routers: This section discusses how to troubleshoot an IKEv2 configuration on a router platform.

“Giving up is the only sure way to fail.”

—Gena Showalter

This chapter covers the following exam objectives:

• 3.0 Troubleshooting using ASDM and CLI

• 3.4 Troubleshooting AnyConnect IKEv2 on ASA and routers

• 3.5 Troubleshooting SSLVPN and Clientless SSLVPN on ASA

• 4.0 Secure Communications Architectures

• 4.4 Recognize VPN technology based on configuration output for remote access VPN solutions

Mastering troubleshooting of clientless and AnyConnect VPNs takes practice. Practice means making mistakes, learning from mistakes, and being able to recognize when a mistake is being made. We believe you don’t truly understand a topic until you not only can make it work but also know why it won’t work. Troubleshooting is a very powerful way to validate that you know a topic such as VPNs, and troubleshooting will come up many times on the SVPN 300-730 exam. Do not overlook this chapter and do not be afraid to fail.

This chapter examines how to troubleshoot the features and critical configuration components used for different VPN solutions covered in previous chapters of this book. It uses a strategy that works across different aspects of a VPN to help you narrow down problems to a specific area of troubleshooting. It is a quick and streamlined troubleshooting strategy. Our goal for this troubleshooting strategy is to help you quickly eliminate obvious wrong answers on the SVPN 300-730 exam by isolating potential correct answers to specific troubleshooting focus domains.

Keep in mind that troubleshooting VPN technology requires knowledge of how the technology works. If you lack knowledge from previous chapters and don’t understand what you are trying to fix, you will most likely find troubleshooting very difficult. For this reason, we highly recommend you first mastering all previous chapters before taking on this last chapter. We see this chapter as a good resource for validating what you have learned by reading this book. The focus will be on troubleshooting remote access VPN technology however most concepts can apply to troubleshooting any form of VPN technology as well.

The SVPN 300-730 exam will challenge you on your ability to recognize common issues. In some cases, the exam will provide a handful of options to choose from. In other cases, it will ask you which packet contains specific information, such as negotiation traffic, that you can use for troubleshooting. The exam might also ask you which command generates particular output or what output results from particular commands. The exam might present a scenario where something isn’t working along with code output that you need to use to determine why the VPN isn’t working.

We highly recommend studying troubleshooting by getting hands-on experience. You must become familiar with what commands, protocols, and technologies do what when used. Do not attempt to memorize a few troubleshooting steps and assume that you have troubleshooting covered. Doing so will likely lead to lost points on the exam and will also not be helpful when you’re troubleshooting real VPN deployments.

“Do I Know This Already?” Quiz

The “Do I Know This Already?” quiz enables you to assess whether you should read the entire chapter. If you miss no more than one of these self-assessment questions, you might want to move ahead to the “Exam Preparation Tasks” section of the chapter. Table 10-1 lists the major headings in this chapter and the “Do I Know This Already?” quiz questions related to the material in each of those sections to help you assess your knowledge of these specific areas. The answers to the “Do I Know This Already?” quiz appear in Appendix A, “Answers to the ‘Do I Know This Already?’ Quizzes.”

1. You are the administrator for a VPN deployment. You have received a ticket saying that a user can't connect to the VPN. You find that the VPN is working fine, but no more users are able to connect. You verify this by using the show vpn-sessiondb command and see that there is room for more users. Which of the following could be causing this problem?

a. The group profile has been misconfigured and is limiting the number of VPN users.

b. The local address pool has been exhausted.

c. There isn't a version of AnyConnect available for the version of the operating system that is trying to connect to the VPN.

d. The ASA is out of memory and can't support the load of another VPN session.

2. Which feature needs to be enabled to support smart tunnel functionality?

a. Java or ActiveX browser support must be enabled.

b. The client must have a proxy configured.

c. Applications can only be UDP based.

d. A smart tunnel hash is mandatory.

3. If clientless VPN bookmarks are grayed out and not working, what is the most likely issue?

a. WebACL is denying access.

b. There is a DNS resolution problem.

c. AAA authorization is not working.

d. The bookmark is missing URL information.

4. When using an XML profile on an ASA, what must match the connection profile?

a. UserGroup

b. HostName

c. HostAddress

d. All of the above

5. What ASA configuration sends all DNS through the SSLVPN tunnel?

a. split-tunnel-dns enable

b. dns tunnel enable

c. dns tunnel full enable

d. split-tunnel-all-dns enabled

6. Which command includes details about data counts, SPI details, and child and parent SA status?

a. show vpn-sessiondb detail anyconnect

b. show run

c. show crypto ikev2 sa

d. show crypto ikev2 sa detail

7. Which method will not help you determine whether the ASA WebVPN service is running?

a. Run the show vpn sessiondb command.

b. Run the show ipsec crypto sa command.

c. View the client TLS event under Monitoring > Logging > Real-time Log Viewer > View.

d. Use the show logging command with logging buffered enabled.

8. Which statement about DfltGrpPolicy is correct?

a. DfltGrpPolicy does not allow any policies by default.

b. DfltGrpPolicy allows administrator SSLVPN client connections by default.

c. DfltGrpPolicy allows all SSLVPN client connections by default.

d. DfltGrpPolicy does not allow SSLVPN client connections by default

9. Which command does not work on a Cisco router for viewing VPN details?

a. debug crypto ikev2

b. show crypto session detail

c. show webvpn anyconnect

d. debug aaa authentication

10. Which ASA VPN show command shows the most details regarding a live VPN session?

a. show vpn-sessiondb detail anyconnect

b. show vpn-sessiondb

c. debug vpn-sessiondb

d. show running-configuration

Foundation Topics

Welcome to the final chapter, which focuses on troubleshooting remote access based VPN deployments. As you read through this chapter, we highly recommend keeping a notepad available. Any time you run into a concept you do not recognize, add it to that notepad as something you will need to master before you attempt to troubleshoot it. Troubleshooting is about making things work properly. If you don’t understand how things work, you will have trouble fixing them.

The SVPN 300-730 exam will challenge you on all aspects of troubleshooting. You will have to validate your understanding of how components work, including what could cause something to break. For example, the exam will show you ASA debug output and ask you to determine what is not working or why something is not working. Any type of troubleshooting topic—from reviewing code to being told about a user experience that is not working—is fair game for the SVPN 300-730 exam. The exam will even expect you to understand how to perform end-to-end troubleshooting of real-world VPN deployments. Know that version 1.1 has 35% of the exam dedicated to troubleshooting questions!

It’s hard to give a firm answer but having hands-on experience is extremely helpful in confirming your understanding of troubleshooting concepts. This chapter works through some of the most common troubleshooting situations we find with our customers as well as situations that are likely to come up on the SVPN 300-730 exam. Consider your ability to troubleshoot as a way to validate that you understand everything you have learned in this book. If you don’t understand any troubleshooting topics covered in this chapter, you should go back to the appropriate earlier chapter and review how things should function before proceeding.

Troubleshooting Clientless SSLVPNs on the ASA

This section takes a step-by-step approach to troubleshooting SSLVPNs. Some of these steps apply to troubleshooting any type of VPN, and other steps are specific to SSLVPNs. The good news about troubleshooting clientless SSLVPNs is that it will likely seem easier than troubleshooting other types of remote connectivity solutions. Clientless SSLVPN technology doesn’t require as many deployment and troubleshooting steps as client VPN deployments. Pay attention to all of the steps and topics in this section; the following sections do not repeat steps that are identical for troubleshooting other topics within this chapter.

Troubleshooting Categories

When troubleshooting SSLVPNs on an ASA, we recommend breaking down the troubleshooting into the following areas:

Step 1. Connectivity: Validate that the client system can reach the ASA.

• Enabled interfaces

• Certificate configuration

• Group URL/alias

Step 2. Login: Troubleshoot any login issues, including validating that the proper connection profile is selected and that the user is able to pass all authentication and authorization steps.

• Connection profile selection

• Authentication

• Authorization

• Group policy selection

Step 3. Web page services: Troubleshoot problems associated with the clientless web page services. The focus in this section is on DNS, plug-ins, and bookmarks.

• DNS configuration on the ASA

• ASA plug-ins

• Bookmarks

Step 4. Application access: Troubleshoot issues related to accessing the application.

• ASA-to-application connectivity

• URL rewrite failures

• Application for port forwarding (needs to point to 127.0.0.1:<port>)

• Application for smart tunneling (wrong SHA-256 or application in the configuration)

During the clientless SSLVPN troubleshooting process, you should step through these four areas to identify where you need to focus your attention. This is a similar approach to troubleshooting the creation of software in the sense that you need to isolate any issue into focus areas so that you can validate and eliminate possible problems as you perform troubleshooting. For example, if a user is unable to log in to a clientless VPN solution, you know your problem falls into the login area, which can be either an authentication, authorization, or group selection problem. Login challenges have three technology components that could be the source of the issue, and you would test each component to further narrow down the problem. Similar logic can be applied as you evaluate sample code or provided troubleshooting situations on the SVPN 300-730 exam; narrowing down the issue can help you quickly eliminate wrong answers. If one answer speaks about URL rewrite failures, that would fall under application access, which has nothing to do with authentication, authorization, or group selection, so it would be easy to drop that as a possible answer for a login failure question. Keep this strategy in mind as you work through this chapter.

These four troubleshooting areas just listed cover most issues related to resolving problems with clientless VPNs on a Cisco ASA, but other issues can exist as well. The Cisco SVPN 300-730 exam blueprint calls out “the need to be able to troubleshoot clientless SSL VPNs on an ASA” and does not specify further details about what is in and out of scope for this topic. Therefore, any configuration related to clientless VPNs could potentially appear on the exam. We highly recommend that you configure your own ASA clientless VPNs solution and work with it until you understand the how the configuration works.

Step 0: SSLVPN Components

Before beginning any troubleshooting, you need to perform step zero, which involves validating the architecture and components. You must understand what you are working with before you take any actions. For clientless VPN technology, there is no VPN client. Hence, the components involve some system attempting to connect to a private network by accessing the ASA. The ASA will authenticate and authorize access, leading to a secure tunnel being established between the ASA and the client. Most organizations use a third-party authentication solution, such as an Active Directory server. The ASA may also host different services, such as, bookmarks or plug-ins. Figure 10-1 shows a basic diagram of the core components you will be troubleshooting in this section.

It’s a great idea to place a simplified version of the architecture in your mind before thinking about how to fix it. Using this approach can help in an exam by enabling you to quickly eliminate any answer involving technology components that are not part of the design spoken about in the question. You can use a similar approach with protocols. Mapping how the technology works in your mind using a simple architecture can help you weed out the obviously wrong answers.

Step 1: Connectivity Troubleshooting

Any troubleshooting action must start with connectivity. Please remember this when dealing with any technology troubleshooting you encounter throughout your time on this planet. Seriously, anything from why your TV is not working to why some security product is not working must have its general connectivity evaluated first when things are not working. People spend countless hours troubleshooting something that is not working and eventually realize that the failure is due to the system not being powered on or a cable being unplugged. We know of a case in which administrators and service providers spent hours troubleshooting a customer’s VPN, only to later find out the headend was unplugged by accident by the local cleaning crew during a late-night cleaning service. You can also think about this in technical terms: Validate Layer 1 before troubleshooting anything else.

Troubleshooting Questions

When troubleshooting connectivity, we recommend starting with basic questions such as these:

• Are the devices powered on?

• Are the cables plugged in?

• Have any security tools been put in place or changed that could prevent connectivity?

• Has anything in the environment changed that could impact connectivity?

• Have there been any recent thunderstorms or reported network outages?

• Has the service provider acknowledged that services are running normally?

• Has anything else occurred that could impact the system?

The most common troubleshooting technique for connectivity is sending a ping between systems that are supposed to be connected. If the ping fails and things worked in the past, the answer to any of the previous questions could identify the reason for the failure. The SVPN 300-730 exam will likely not test you on cables failing or systems being powered off, but real-world deployments will have these issues, and overlooking troubleshooting related to these basic connectivity problems can to countless hours wasted on more advanced troubleshooting techniques. We don’t cover basic connectivity troubleshooting details again in this chapter, but you should keep in mind that this is always the first step, regardless of the VPN—or other technology—being tested.

Exam-Focused Connectivity Troubleshooting

The connectivity troubleshooting you will encounter on the SVPN 300-730 exam will likely include failures in protocols or things that can be configured within the client or ASA rather than a cable being unplugged. For the exam, you can typically assume that power and cable issues are not the problem; the exam is more likely to test you on configuration problems. For example, the VPN user might be able to reach the ASA, but the ASA does not have a certificate or group configured correctly, and thus the user has basic connectivity issues. In this case, as with basic connectivity testing, you would start by sending a ping to the outside IP address interface. Additional steps for connectivity troubleshooting include validating that the interfaces of all used connections are up, pinging from the client side, interviewing the client for details regarding the issue they are seeing, and validating that the Cisco ASA can ping the next hop router or an IP address hosted on the Internet. Finally, you should also check licensing as it is possible that you have run into a situation where the permitted user count is oversubscribed.

ASA WebVPN Service

After you validate connectivity by using ping and preforming other basic connectivity testing, you should determine whether the ASA WebVPN service is running. Because the ASA is listening on the outside interface for the inbound 443 request, you must determine whether that service is functioning. If it is not functioning, users will not be able to connect to the ASA and will receive a connection failure error. We recommend checking whether a service such as WebVPN is running after performing basic connectivity for all VPN troubleshooting.

You need to check the ASA configuration to ensure that the WebVPN service is up and that it is not configured for another port. An easy way to do this is by using the ASDM monitor page or from the CLI, using the show run webvpn command, as shown in Figure 10-3. Two other options are the commands show vpn-sessiondb and show asp table socket. When a client connects to the ASA, in ASDM, you can see client TLS events under Monitoring > Logging > Real-time Log Viewer > View. With logging to the internal buffer enabled (via the command logging buffered severity_level), you can also see the same output when using the command show logging. Be sure to be familiar with all of these options for the SVPN 300-730 exam.

Troubleshooting Certificates

One reason users may have connectivity issues could be the certificate configuration. A common issue is that a certificate has expired or been revoked. Besides these obvious certificate issues, the certificate problem could be related to how the certificate was configured. You need to make sure that a certificate is signed and that the certificate’s common name is the fully qualified domain name (FQDN) used by users to connect to the ASA. You need to validate that the signed certificate is imported into the ASA. Chapter 9, “AnyConnect VPNs on the ASA and IOS,” provides an example of using the crypto ca import command to import a base-64-encoded certificate. During installation of the signed certificate, the ASA provides output informing you of the FQDN used by the certificate. A mismatch between the FQDN and what users use to connect to the ASA can cause connectivity issues. Be sure to verify that the ASA FQDN is the same as on the certificate. We have run into this issue a handful of times when new VPNs are being set up in customer environments.

Applied Certificates

Another issue to check is whether the certificate is applied to the interface used to terminate the SSLVPN. Using the command ssl trust-point from the command line applies the certificate to the correct interface. If this is not done, the proper certificate won’t be used when a VPN is established, and this can lead to connectivity issues. This same issue can lead to inbound VPN user connections receiving a certificate warning. This warning could mean that either a self-signed certificate was applied to the interface or, during the installation, something failed; for example, the proper certificate might not have been applied to the interface.

Full Certificate Chain

One final certificate validation is verifying that the full certificate chain is loaded on the ASA. A certificate can be installed on an ASA using either the command line or ASDM. The command for installing a certificate is crypto ca authenticate ssl-trustpoint, and you can install an SSL certificate in ASDM by going to Configuration > Remote Access VPN > Advanced > SSL Settings. A PKCS12 certificate can also be used. Regardless of what certificate is used, the entire certificate chain must be copied, or you will run into problems. You can verify the installed certificates in ASDM by going to Configuration > Remote Access VPN > Certificate Management, or you can use the command line to run the command show crypto ca certificate.

Correct Certificate

The next step in troubleshooting certificates is to verify that the proper certificate is being used by the hosts attempting to connect to the VPN. To do this, first connect to the WebVPN through a web browser using https:// followed by the FQDN. After you request the certificate, double-click the lock icon that appears in the lower-right corner of the WebVPN login page to bring up the installed certificate information.

Certificate Debug Commands

If you believe you have a potential certificate failure, you can run either debug crypto ca 255, debug crypto ca message 255, or debug crypto ca transaction 255 to view details about the potential failure. One example of a common problem is the error message “Untrusted certificate warning when using a valid third-party SSL certificate on the external interface.” This error can occur when an RSA key pair is used with the certificate. By default, ASA software version 4(1) and onward have all ECDSA and RSA ciphers enabled by default, and the strongest cipher (usually an ECDSA cipher) is used for negotiation. This could cause the ASA to present a self-signed certificate instead of the currently configured RSA-based certificate, and you would see this in the debug output. A fix for this specific problem is to disable ECDSA ciphers and test.

The capture Command

The last suggestion for troubleshooting connectivity is to use the capture command on the ASA. The capture command creates a .zip file encrypted with the password koleso that includes HTML contents of clientless web pages that the client accesses. This is useful for troubleshooting websites that do not display properly without having to access the user’s host system. The syntax for this command is capture webvpn-issue type webvpn user user. For this example, the output name of the capture command is webvpn-issue, and it is located in the default capture directory on the ASA by name.

Connectivity Troubleshooting Summary

The following are the key concepts for troubleshooting connectivity problems:

• Always start by troubleshooting basic connectivity possibilities, including power and cabling.

• The simplest way to validate connectivity is by using ping.

• Validate VPN licenses by using the show vpn-sessiondb license-summary command.

• Validate that the ASA WebVPN service is running on the ASA by using the command show run webvpn or using the ASA GUI.

• Validate that the entire certificate chain is imported into the ASA, and ensure that it has a FQDN and the certificate is properly applied to the correct interface.

• To get a general view of connectivity, use the capture webvpn-issue type webvpn user user1 command.

Step 2: Login Troubleshooting

If the user is able to reach the HTML landing page for the ASA, you know that connectivity works. The next area of troubleshooting focuses on the user logging in to the VPN. A user who can’t log in to the VPN will likely see an error or failure message indicating that there is an issue with group selection, authentication, or authorization. In such a case, you must determine why the user is unable to log in after successfully connecting to the landing page. Troubleshooting the login process for an SSLVPN can be broken down into a few areas: connection profile selection, group URL, authentication, and authorization.

Connection Profile Group URL

A group URL can be defined under a connection profile. Using a group URL enables the remote client to automatically select a connection profile, simplifying the process of remote users connecting to the VPN. This is different from the alias concept, but URLs and aliases can use the same names. An example of a group URL would be https://ciscoasa.example.com/clientless. This would take the user directly to a WebVPN login page for that specific connection profile. The use of a connection profile URL with an FQDN requires the appropriate name to be defined in DNS for users to reach it. You need to look at this first when troubleshooting a group URL. You must validate that the URL is a FQDN that users can reach by using ping or nslookup. You also must validate that users are going to the correct URL by asking them for the specific URL they are attempting to access. Any of these issues would lead to a URL not working.

Say that you configured your ASA to support a URL address for a group of contractors, and you provide those contractors a customized URL for their web VPN access needs. You inform the contractors to log in to their customized web portal by using the URL https://vpn-asa1.example.com/contractor. (We cover how to build this type of setup in Chapter 9, and you need to understand this configuration because the SVPN 300-730 exam might question you on both building and troubleshooting an SSLVPN leveraging a customized group URL.) If the wrong URL is provided to the VPN user, or if the user isn’t authorized to use that connection profile, the user might be unable to access the VPN. As part of troubleshooting, you would need to ask for the URL that was used and where the contractor was connecting from. You need to test the URL and might also need to access the network used by the contractor to validate that you can access the URL from the user’s point of view. (If you need to test connectivity from the user side, see the connectivity steps described earlier in this chapter.)

Viewing Group URLs

To view the group URLs configured for a particular connection profile, open the connection profile and navigate to Advanced > Group Alias/Group URL. Figure 10-4 shows an example of group URLs configurated for the EMPLOYEE_CONNECTION connection profile. Notice the three different capitalizations of employees: EMPLOYEES, Employees, and employees. The URL is case sensitive, and so the three most common capitalizations of employees were used to ensure a more consistent user experience if users enter the URL manually.

Profile Selection

A user who gets to the SSLVPN login window in a browser might be given the option to select a group at the login prompt. Whether a user is presented with the option to select a group depends on two configuration options. First, the option Allow Users to Select Connection, Identified by Alias in the Table Above, at Login Page must be enabled within ASDM. Second, a connection alias must be configured in the connection profile. Without either of these being enabled/present, the user will not have the option to select the specific connection profile at login. Why is this so important? A user who selects the incorrect connection profile might not be able to authenticate or may be denied access, in which case you, as the VPN administrator, will receive a troubleshooting ticket.

For example, let’s say that a test user is configured as a local user on the ASA and is instructed to use the LOCAL_USER_PROFILE connection profile to log in. Instead, the user chooses the RADIUS_USER_PROFILE connection profile and tries to log in, but their username and password are rejected. Why might that be? The RADIUS_USER_PROFILE, as the name implies, is configured to use RADIUS to authenticate users, but the user is not set up on the RADIUS server. This user is configured as a local user on the ASA and hence must use a connection profile that uses the local user database for authentication to an SSLVPN group that does not support clientless SSLVPN access. When the test user attempts to log in to WebVPN, that user cannot log in using any group policy via the web interface and fails due to using the default group policy, which, by default, does not support SSLVPN clients.

Authentication

If connectivity and the connection profile look correct, the next step is to validate authentication. Authentication means showing that something is true, genuine, or valid. When troubleshooting authentication, you essentially need to validate that the users can prove who they are. You could use a local database for managing users, but organizations typically use external authentication solutions, such as Microsoft Active Directory (LDAP), RADIUS, or TACACS+. Troubleshooting these types of login authentication issues may involve simply determining whether AAA server groups are working properly and whether the user ID is in the remote authentication system.

ASA Authentication Testing

A Cisco ASA includes features to test authentication with a user ID and password. You enter a user ID and submit it to the ASA to test the authentication process. For example, a system administrator could first test their own ID to determine if LDAP is configured properly and communicating with the Microsoft AD server. If that works, the next step would be to perform a similar test for the user account that is having problems. We recommend using this process as a way to validate whether a username ID would work if used for connecting to your VPN setup.

Debug ASA to Authentication System

If you perform an initial login validation test and basic authentication is not working, the next step would be to test/debug the process from the ASA to the authentication system. You can enable LDAP debugging by using the command debug LDAP 255 on the ASA. For testing and troubleshooting RADIUS, you can start troubleshooting by using the command-line test command test aaa-server authentication Radius-Server-Group host 1.2.3.4 username johndoe password cisco123. If this test fails, you can next try the debug radius aaa-request command. Each authentication protocol supported on the ASA includes a test and debug process, and the output shows any potential issues.

Figure 10-5 shows an example of output from the debug LDAP 255 command after a user attempts to log in to the VPN. Notice some key points that are highlighted, including debugging being enabled, the authentication of the user “doctor” being successful, and details about the doctor’s profile. The command debug RADIUS 255 is another command you might use. (Keep in mind that these commands provide more output than is shown in Figure 10-5.)

Authorization

Troubleshooting the authorization process is like troubleshooting the authentication process, but the focus is on validating what authenticated users are permitted to access. The troubleshooting process depends on what protocol you are using for external authorization. This chapter focuses on one protocol, but Cisco provides several technical guides on troubleshooting each of the authorization methods supported by the ASA. The process is basically the same, regardless of the authorization protocol used, but the commands required for troubleshooting are slightly different.

Authorization Debugging

The Cisco ASA supports several methods for applying user authorization attributes to clientless VPN connections. For example, you can configure the Cisco ASA to use the group name supplied by LDAP (Microsoft AD) attributes. As mentioned earlier, if the group attributes are incorrect, the user cannot log in. An easy way to debug the credentials is by using the debug LDAP 255 command to determine whether the group attribute is mapping correctly. You can simply enable debugging and attempt to log in to the VPN to view any error messages. Figure 10-5 shows an example of using the debug LDAP 255 command and viewing output after a user attempts to connect to the VPN solution.

Group Policy

Closely related to authorization is the group policy configured for a connection profile and/or user and the access permissions it grants to a user. Examples of group policy settings include access hours, simultaneous logins, and maximum connection time, just to name a few. Each of these settings could cause a user to either fail to log in or lock out a user who falls outside of what is permitted by the group policy configuration. Troubleshooting group policies must include validating the group configuration and matching it against the user’s expectations, which can be done by simply accessing the policy and noting what features are enforced. You need to collect information on the situation, including when the user logged in, what user account is being used, and other data points that you validate against the group configuration. Figure 10-6 shows an example of the group selection configuration options.

Group Policy Validation Using CLI

You can view the same information by using the Cisco ASA command line. To focus on group policy information, you can run the show running-configuration all group-policy command to see how any group policy has been configured. You can also add the group policy of interest to the end of the command. For example, Figure 10-7 displays the DfltGrpPolicy settings.

Login Troubleshooting Summary

The following are the key concepts for troubleshooting login problems:

• DfltGrpPolicy does not allow SSLVPN client connections.

• Testing a group URL involves validating that the URL is correct, ensuring that it has a FQDN, and making sure the user is permitted to log in with the associated connection profile.

• For Activate Directory authentication testing, use the Cisco ASA to test authentication such as the user ID and password.

• At the Cisco ASA CLI, use the command debug LDAP 255 or debug RADIUS 255 to debug LDAP problems. Enable debugging and attempt to log in to the VPN.

• For RADIUS, use the command test aaa-server authentication Radius-Server-Group host x.x.x.x username johndoe password cisco123 to validate a login and password.

• If RADIUS validation fails to work, run the debug radius 255 command and view the error logs.

• Troubleshooting group policies must include validating the group configuration and matching it against the user’s expectations.

Step 3: Clientless WebVPN Service Issues

Earlier in this section, you saw an example of validating whether the ASA WebVPN services are running. That example does not look at how those services are functioning. Remember that you first want to ensure that connectivity works and that login issues have been resolved before you dive into how the VPN works.

WebVPN services problems are related to the browser on the client machine. One problem a user could complain about is not being able to see the WebVPN page, even though they can successfully connect to it. One possible problem that could cause this issue is the user’s browser not trusting the web page provided by the ASA due to a corporate policy restriction or another limitation on the user’s system. Troubleshooting this particular problem will most often have to be done from the user system. However, you could use another system to verify that the service is functioning correctly before moving to performing similar tests on the impacted user’s system. If you can access the user’s system, you will want to validate that the ASA’s provided web page is included in the browser’s trusted zone as well as that cookies are enabled. You will also want to try clearing the browser cache and Java cache before trying to connect to the ASA WebVPN page again. In short, you want to focus troubleshooting on the user’s browser that is experiencing problems first before proceeding with troubleshooting the ASA side of the service. Figure 10-8 shows the browser configuration options on a Chrome browser.

Validating WebVPN Service Details

If you believe the WebVPN service should be working and have tested the VPN user’s web browser, but there are still problems associated with the service, you need to look more closely at how the service is configured and running on the ASA side. One place to start is by using the show vpn-sessiondb detail webvpn command. Figure 10-9 provides an example of using this command and demonstrates that the user vpnuser is connected to the ASA. Notice that the other fields in this output also provide useful troubleshooting information. For example, the browser type is included, which might help determine if the user is using a version that is known to have problems.

In addition, the authentication mode (Auth Mode) and group policy might provide helpful information if a user is able to log in to the VPN but cannot access required services. We highly recommend being familiar with the show vpn-sessionsdb detail webvpn output.

WebVPN Debugging

If you need to take a deeper look at troubleshooting the WebVPN service, you can use the debug command. The command debug webvpn condition user user can help you see specifics related to a user or group. If an ASA has hundreds of inbound WebVPN connections but a single user is having a problem, then this might be the option to employ to look for error messages. You could run this command and ask the user to attempt to connect to the VPN. The output should include error messages that might lead you back to the ASA or user side for further troubleshooting.

Validating DNS Configuration

Another possible problem with WebVPN services could involve the DNS configuration on the Cisco ASA. DNS is a good starting point if connectivity to the Cisco ASA works but the user can’t access a protected application. Remember that, technically, the user’s application requests initiate from the ASA, so you need to make sure that the ASA can resolve names. To validate that the ASA can resolve names, you need to check the ASA configuration for the critical command dns domain-lookup inside. In addition, you need to make sure that the IP address associated with the DNS server is correct by looking at the dns name-server x.x.x.x setting. You might want to add a second DNS name server to provide a level of fault tolerance if you suspect DNS issues. If the WebVPN client application is accessing applications through a DMZ interface, you should make sure DNS lookup is also enabled in the ASA. You can see all of these settings by using the show run or show run dns command to view the ASA configuration. Figure 10-10 shows output of the show run dns command.

ASA Plug-ins

The Cisco ASA supports multiple plug-ins related to specific applications. These plug-ins support RDP, VNC, Citrix, SSH, and Telnet. Keep in mind that each of these plug-ins presents a potential unique issue to troubleshoot. Best practice regarding plug-in maintenance is keeping all plug-ins updated and understanding the pieces that they are dependent on for their functionality. An example of a dependency is that some plug-ins require Java and must have the correct Java feature set for the plug-in to function properly. If a VPN user’s Java client is out of date or the VPN user is using an old version of Java, they could experience problems unrelated to how the ASA is configured; this would be a host problem. If you run into a dependency problem such as one with Java, we recommend removing Java and reinstalling the latest version on the VPN user’s system.

Troubleshooting ASA plug-ins involves isolating the problem to the plug-in and validating dependencies for the plug-in. Then you can troubleshoot from the user’s side as the issue is likely related to the user being unable to support the plug-in. Figure 10-11 shows plug-in options.

You can also view what WebVPN plug-ins are being used by leveraging the ASA command line. Run the command show import webvpn plug-in and look to see what plug-ins are listed.

Bookmarks

You saw how to configure bookmarks on an ASA in Chapter 9. Bookmarks are designed to help enable easy setup of applications for VPN clients. If a user complains that a bookmark isn’t working, you should check the group policy associated with the user. It is possible that the user does not have the correct rights to see the bookmarks list. Next, you need to make sure the bookmarks list is not empty since that would cause the bookmark to not work. You can validate a bookmarks list by using the command export webvpn url-list url-list-name stdout. We recommend starting with these two items as they are the most common reasons for bookmark access problems. Figure 10-12 shows an example of exporting bookmarks in a file named CONTRACTOR-BOOKMARKS.

DAP and Bookmarks

If the bookmarks list still does not appear on the VPN user’s system but looks to be available for them to use, try checking the dynamic access policy (DAP) settings. A DAP rule could be removing the bookmarks list from the user or group, leading to the bookmarks not being available. To validate whether DAP is the culprit behind the bookmark issue, you can execute the command debug dap trace.

DNS and Bookmarks

If users complain that a bookmarks list is available but grayed out, you need to ensure that the DNS server is reachable from the ASA. You need to validate that the ASA can ping the website by name to rule out this possible issue. If a ping does not work, you need to verify that the ASA has the correct DNS configuration.

WebVPN Services Troubleshooting Summary

The following are the key concepts for troubleshooting WebVPN services problems:

• WebVPN services problems are typically related to the browser on the client machine. You should start by troubleshooting the user’s side before moving to validating that the ASA is configured properly.

• The show vpn-sessiondb detail webvpn command can be used to view the WebVPN service status details.

• You should validate that the DNS configuration exists on the ASA by using dns domain-lookup inside and dns name-server x.x.x.x with the show run command.

• ASA plug-in problems can be isolated to the plug-in and its dependencies. For example, a browser might not support a plug-in dependency such as Java.

• When troubleshooting bookmarks, make sure a bookmark exists and ensure that the user has privileges to view the bookmark. You also need to ensure that a DAP rule isn’t causing the problem by using the debug dap trace command.

Step 4: Application Access

The final area of focus for clientless VPN troubleshooting is application access problems. Issues associated with applications launching through a clientless VPN can be challenging to solve because the ASA is proxying the connectivity, and the ASA HTTP server interacts with the authentication subsystem to authenticate the user for the specific application. When troubleshooting application access, Cisco recommends that you review the limitations for clientless SSLVPN to determine whether the application and its associated security model will function properly. This way, you can avoid wasting time troubleshooting something that Cisco has already identified as not working. In short, you should start by researching the application’s support through the clientless SSLVPN before taking any other troubleshooting actions.

ASA-to-Application Connectivity

We started off troubleshooting problems with a focus on connectivity. If you find that a VPN is working but the application is not, you should test the connectivity between the ASA and the application. The key for troubleshooting connectivity is to determine where the error is coming from. First, validate that the ASA and the application server have connectivity by using ping. Next, determine if the local web client browser can access the application by using the tactics covered earlier in this chapter regarding user browser troubleshooting. If the browser settings are correct, try to clear the browser cache and the Java cache. You can also disable the ASA’s cache at the CLI. The commands to use are webvpn followed by cache, as shown in Figure 10-13. Figure 10-13 shows the configuration options, including disabling cache.

Application-to-ASA Connectivity with Port Forwarding

Recall from Chapter 8, “Clientless Remote-Access SSLVPNs on the ASA,” that port forwarding requires a change in how users connect to applications. For example, instead of connecting to hr.example.com on port 3389, the user instead might need to connect to 127.0.0.1:<port forwarding port>. You can see port forwarding in the window shown in Figure 10-14.

From a troubleshooting perspective, if port forwarding is in use, you need to ensure that the user is connecting to the proper local IP address and port. Otherwise, connectivity will fail.

Application Troubleshooting Summary

The following are the key concepts for troubleshooting application problems:

• Application troubleshooting starts with researching the application and validating that it is supported by Cisco.

• You need to validate connectivity between the ASA and the application as well as between the user’s browser and the application.

• You should attempt to troubleshoot the user’s browser problems by using steps previously covered as well as by clearing the browser cache.

Troubleshooting AnyConnect SSLVPNs on the ASA

This section focuses on troubleshooting an AnyConnect SSLVPN deployed using a Cisco ASA. As in the first section of the chapter, on troubleshooting clientless VPNs, we use a step-by-step approach in this section. The major difference for this section and the remaining sections is that now we are shifting to client-based VPN architecture; in this case, the client is Cisco AnyConnect. Adding a VPN client opens up new areas to troubleshoot, including how the customer obtains the client, ensuring that the client is supported by the host system, validating that the client is functioning properly, and using the client to obtain data for troubleshooting. Figure 10-15 shows a basic AnyConnect SSL deployment that includes a VPN client, a Cisco ASA acting as the VPN concentrator, and an external identity database. Once again, try to keep a simple diagram like this in your mind as you approach any SVPN 300-730 exam troubleshooting questions as doing so will help you eliminate obviously wrong answers.

Some of the troubleshooting concepts covered in the first section of this chapter apply to all VPN deployments. To avoid being repetitive, in this section we summarize those concepts and focus on what is either new or specific to AnyConnect SSLVPN deployments. You will also find some of these repeating concepts in our recommended summary for troubleshooting AnyConnect SSLVPN deployments since we want to keep our troubleshooting process intact.

Troubleshooting AnyConnect SSLVPNs involves the following steps:

Step 1. Connectivity: Confirm all aspects of connectivity between the user and the VPN provider (in this case, the Cisco ASA).

Step 2. Login: Troubleshoot any login issues, including validating that the proper user group is selected and that the user is able to pass all authentication and authorization steps.

Step 3. Network access: Review all aspects of how network access is provided by the VPN. Many of these topics are different than with the clientless VPN troubleshooting process:

• Address pool

• Routing problems

• DNS

• Browser proxy

• AnyConnect client

• NAT

• Traffic filters

Step 4. Diagnostic and Report Tool (DART): Leverage the Cisco diagnostic tool for AnyConnect.

Step 5. VPN diagnostic commands: If the VPN is working but certain configurations are possibly not correct, run diagnostic commands to view details about how the VPN is running.

Step 6. Application access: Troubleshoot issues related to accessing the application.

• ASA-to-Application connectivity

Let’s start from the top of the list with troubleshooting connectivity.

Step 1: Connectivity Troubleshooting

As you have already seen in this chapter, the first step for any troubleshooting process is to examine connectivity. The first section of this chapter covers key points to validate, including communication between systems, the status of interfaces involved with the connectivity, possible certificate issues, license problems, and anything else that prevents connectivity from occurring. We recommend working up the OSI model from Layer 1 (the physical layer) to ensure that you can reach the VPN application.

If you find connectivity problems by using commands like ping but are unsure of the cause, you can use the command debug webvpn anyconnect. Debugging enables you to view error messages as you perform connectivity testing.

Step 2: Login Troubleshooting

As with clientless, when troubleshooting a client-based VPN, you focus on ensuring that accounts are created and authorized to access the VPN. You can test this by using an administration account or using the client’s account and verifying that services are functioning. Debugging can also be used to view login problems such as the wrong password being used or a reference to the wrong access group. You should follow the same troubleshooting steps presented earlier in this chapter, starting with basic authentication verification and moving to group member problems. Do not proceed with further troubleshooting until you have ruled out connectivity and login issues as possible causes of an SSLVPN problem.

Step 3: Network Access Troubleshooting

After you validate that connectivity and login aspects of the VPN are functioning as expected, you should move on to troubleshooting network access. Unlike with a clientless VPN, the AnyConnect VPN has potential complexity around routing IP packets to the correct destination. This means you need to take different steps for troubleshooting, including looking at how AnyConnect establishes the VPN service and potential NAT or DNS issues.

This section starts by troubleshooting using ASDM but also shows the same steps using the ASA command line. The SVPN 300-730 exam will likely have the command line version of steps; however, real-world troubleshooting can be accomplished using both the command line and the GUI, so we cover both approaches here.

AnyConnect Enabled

With ASDM, navigate to Configuration > Remote Access VPN > Network (Client) Access. Verify that AnyConnect is enabled via the appropriate check boxes. You might also want to validate that DTLS is enabled, as it typically provides better performance, especially for jitter-sensitive applications such as voice. If the VPN action check boxes under the SSL Access section for Interfaces are not checked, you are not allowing network access and so will block users. Make sure the connection profile you are using also has SSL enabled. Figure 10-16 shows an example of some of the check boxes you need to examine.

You can validate the same information by using the ASA command line and running the command show running-configuration webvpn. The output enables you to verify whether the outside interface has been enabled for WebVPN, what profiles have been set up, whether AnyConnect is enabled, and details on the smart tunnels being used (see Figure 10-17).

Group Policy Configuration

After you make sure all the correct options are enabled, you need to look at the group policy configuration. Think about what the VPN concentrator (ASA) needs to provide a client. When you create a VPN, you are essentially providing a new network configuration, which means a unique IP address that is separate from the client. To validate this, you go to Configuration > Remote Access VPN > Network (Client) Access > Group Policies. The examples in this section focus on the default group policy DfltGrpPolicy (the system default). You can also view group policies by using the command line and running the show running-configuration group-policy command. While in ASDM, double-clicking a policy brings up a separate window related to the AnyConnect client configuration.

The following are some of the areas where you need to focus your troubleshooting:

• Address pool

• Routing problems

• DNS

• Browser proxy

• AnyConnect client

• NAT problem

• Traffic filters

Let’s work through each of these focus areas more closely to see they relate to potential problems you might need to troubleshoot.

Address Pool

When you open a policy and begin troubleshooting, you should validate the address pool. Make sure you click the Select button and verify what IP address range is being used. If the range being used is too small to support the VPN deployment, a client might not be able to get an IP address, and thus communication will fail due to lack of available IP addresses. You also must ensure that the IP subnet mask and the address block used by the address pool are in the headend routing table and that the routing table indicates the correct location for the client. We have been at organizations troubleshooting VPN deployment issues and found the organizations using one-way routing. When viewing traffic in this situation, you see that the VPN client is transmitting traffic, but nothing is returning. This behavior would be a good indication that the headend (the terminating side of the VPN tunnel) does not know how to return packets to the client. Figure 10-18 shows an address pool configuration example for DfltGrpPolicy.

Validating the Address Pool

You can quickly find any address pool from the ASA command line by issuing the command show running-config tunnel group. This command shows details for any connection profile, including associated address pools. Alternatively, you can simply filter on address pools by using the command show running-configuration | include address-pool followed by show running-configuration | include THE_VPN_POOL_Name to see details regarding the IP address range within the VPN pool. For example, the following output could come from our small lab (although a real-world deployment would have a range of IP addresses):

ASAv# show running-configuration | include address-pool
 address-pool value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL

You would validate this address pool by using a show command and ip local pool or using the command DCLOUD-VPN-POOL, as in this example:

ASAv# show running-configuration | include DCLOUD-VPN-POOL
ip local pool DCLOUD-VPN-POOL 198.19.19.100 mask 255.255.255.255
 address-pools value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL
 address-pool value DCLOUD-VPN-POOL

Routing Problems

After you confirm that the address pool is correct, your next area of focus should be how traffic is routed. Routing table problems can take many forms, as you have already begun to see. Another common issue is that the client may be receiving a split tunneling policy, but the policy does not include the correct address range. You should validate what IP range is provided by going under the connection profile default group policy found under the Advanced settings within the group policy. Here, you need to verify what type of tunnel is being configured and view details about the network list, which shows what networks are passing through the tunnel. If you click the Managed button next to the network list that is being used, you can see what access control list the client will receive when this group policy is applied. You need to verify that the tunnel policy isn’t applying an access list policy that blocks traffic. Figure 10-19 shows an example of validating the access list for a tunnel policy.

Use the show running-configuration tunnel-group command to view the connection profile configuration details using the ASA command line and use the show running-configuration group-policy command to view details. To view access list details, you can use the command show running-configuration access-list, which lists all the access lists configured. You could also use the tunnel-group command to see details about the connection profile you are interested in including and to identify any configured tunnel policies. Next, you run the show command for the specific group policy to get an idea of any associated access lists. Finally, you run the show access-list command to view details for the related access lists.

DNS Troubleshooting

The DNS configuration of the AnyConnect client permits clients to resolve internal IP addresses by using the tunnel connection. This is obviously important and needs to be validated. There are also situations, such as when using split tunneling, in which you do not want all DNS lookup traffic to come through the VPN tunnel. You need to validate which approach is being used and match it to your desired DNS results. Validating the DNS being used could be as simple as using the client ping command from the CLI and viewing the results.

To view how DNS is being configured, you need to view whether split DNS is being used or whether all DNS lookups are being sent through a tunnel. Split DNS enables a client to send specific DNS queries across the tunnel, and all others uses an outside DNS server. When all DNS lookups are being sent through a tunnel, all DNS queries are sent to the organization’s DNS server. Figure 10-20 shows where this is configured under the Group Policy > Advanced > Split Tunneling.

You can view the same information by using the show running-configuration group-policy command, as shown in Figure 10-21. You need to look for the line split-tunnel-all-dns and see if it is enabled or disabled.

DNS Split Tunnel Range

You also need to make sure the DNS server at the headend of the VPN architecture is within the split tunnel range. If it is not, DNS resolution will fail. Verify this by viewing the address pool range along with the network list used by the split tunnel policy, as already discussed. In addition, the DNS server could block DNS requests from the client because of corporate or other existing policies. This issue would relate back to validating connectivity, including whether security tools are preventing the connection.

Browser Proxy

The browser proxy settings of the AnyConnect client are designed to push web browser proxy settings and force the client to use specific proxy server settings. There are a variety of options for this attribute, and troubleshooting this should be considered when a user is unable to browse websites. You can validate this under the group policy by viewing the browser proxy settings and reviewing whether a browser proxy is being used as well as what the address is for that proxy. Figure 10-22 shows the configuration page with a dummy proxy placed as the proxy server address.

You can also view this information by running the show running-configuration group-policy command. Figure 10-23 shows an example for the DfltGrpPolicy and viewing the same proxy server as shown in Figure 10-22 by using the ASA command line.

NAT Problem

If NAT is not done correctly, it will quickly lead to trouble. Remember that with NAT, the client is sending traffic, and the NAT policy is changing the packet on the way to the network or out from the network. To validate NAT, you can use a tool provided by Cisco, known as Packet Tracer, to view the NAT logic. You can use this tool from the ASA command line, and it is also embedded in ASDM. You can find the tool in ASDM under the Tools menu or by going to Configuration > Firewall > Access Rules and right-clicking an IP address range. You then choose Packet Tracer, as shown in Figure 10-24.

capture Command

You can also use the ASA command line to capture packets for troubleshooting. You run the command capture capture_name interface interface_name, where capture_name is the name of the capture, and interface_name is the name of interface used to perform the capture. You end the command with the IP address range you want to collect.

For example, you could collect traffic to a capture named capin, via the inside interface, matching the source IP address 198.19.255.1, the mask 255.255.255.255, the match destination IP address 198.18.133.1, and the mask 255.255.255.255 as follows:

ASA# capture capin interface inside match ip 198.19.255.1 255.255.255.255 198.18.133.1 
255.255.255.255

Another example for collecting traffic from the outside interface would be as follows:

ASA# capture capout interface outside match ip 198.19.255.1 255.255.255.255 198.18.133.1 
255.255.255.255

When you run a capture command, the ASA begins collecting traffic flow until you stop it. To stop the ASA from collecting packets, you run the command no capture capture_name interface interface_name. To view the details of what was captured, run the show capture capture_name command.

capture Command Options

The following are some of the options available with the capture command that you need to be familiar with for the SVPN 300-730 exam:

• asa_dataplane: Captures packets on the ASA backplane that pass between the ASA and a module that uses the backplane, such as the ASA CX or IPS module.

• asp-drop drop-code: Captures packets that are dropped by the accelerated security path. drop-code specifies the type of traffic that is dropped by the accelerated security path.

• ethernet-type type: Selects an Ethernet type to capture. Supported Ethernet types include 8021Q, ARP, IP, IP6, IPX, LACP, PPPOED, PPPOES, RARP, and VLAN.

• real-time: Displays the captured packets continuously in real time. To terminate a real-time packet capture, press Ctrl+C. To permanently remove the capture, use the no form of this command. This option is not supported when you use the cluster exec capture command.

• trace: Traces the captured packets in a manner similar to the ASA Packet Tracer feature.

• ikev1/ikev2: Captures only IKEv1 or IKEv2 protocol information.

• isakmp: Captures Internet Security Association and Key Management Protocol (ISAKMP) traffic for VPN connections. The ISAKMP subsystem does not have access to the upper-layer protocols. The capture is a pseudo capture, with the physical, IP, and UDP layers combined together in order to satisfy a PCAP parser. The peer addresses are obtained from the SA exchange and are stored in the IP layer.

• lacp: Captures Link Aggregation Control Protocol (LACP) traffic. If configured, the interface name is the physical interface name. This might be useful when you work with EtherChannel in order to identify the present behavior of LACP.

• tls-proxy: Captures decrypted inbound and outbound data from the Transport Layer Security (TLS) proxy on one or more interfaces.

• webvpn: Captures WebVPN data for a specific WebVPN connection.

For the SVPN 300-730 exam, you will likely not have to know how to run Packet Tracer. However, you could see the output of one of these versions of the command-line Packet Tracer command and asked to interpret it. The goal when viewing the output of a packet trace in regard to NAT is to validate how NAT is occurring to ensure traffic is being routed properly. We recommend keeping this skill in mind as packet tracing can also be used for troubleshooting many other network-related issues.

Traffic Filters

The ASA provides a way of filtering traffic that traverses the VPN tunnel, which is a good thing but can also cause some problems if this feature is not properly configured. Traffic filters are configured on the group policy and consist of rules that determine whether to permit or deny tunneled data as it transverses the ASA. You can filter on source IP address, destination IP address, or even protocol. Administrators use traffic filters as essentially ACLs in order to block or permit various types of traffic but from the VPN policy level. It is important to validate what traffic filter policies are in place if a user complains they are unable to access a specific resource, such as an application.

You configure a traffic filter by first building an ACL and assigning the ACL to a group policy. Next, you assign the group policy to the connection profile used by the VPN and that will force VPN traffic through your filter. An example of a configuration would involve creating an ACL such as one called VPN-FILTER and a group policy called VPN-FILTERPOLICY and then assigning default traffic through the group policy, as shown in the following configuration example:

access-list VPN-FILTER permit ip 192.168.1.0 255.255.255.0 10.1.1.0 255.255.255.0
group-policy VPN-FILTERPOLICY internal
group-policy VPN-FILTERPOLICY attribute
 vpn-filter value VPN-FILTER

Troubleshooting Traffic Filters

Troubleshooting involves validating the ACLs used in the policy. Essentially, an ACL has an option to permit or deny any resources that are part of what is being permitted or denied. The following is a general ACL skeleton representing what you would need to view as you verify what should be permitted or denied by any traffic filters:

access-list VPN-FILTER permit <remote-IP> [remote-Port] <local-IP> [local-Port]

Network Access Troubleshooting Summary

The following are the key concepts for troubleshooting network access problems:

• Start network access troubleshooting by validating that AnyConnect is enabled, DTLS is enabled, SSL is allowed out of the ASA, and the connection profile being used has SSL enabled.

• Next move to the group policy configuration and verify that the address pool(s) and routing are set up correctly, including whether and how split tunneling is being used.

• Validate whether split DNS or whether all DNS lookups are being sent through a tunnel, and make sure configuration settings are correct.

• Verify whether a browser proxy is enabled and review the AnyConnect client settings that are associated with the VPN policy being used.

• Check for NAT and traffic filtering problems by running Packet Tracker and validating any ACLs that are being used by the VPN policy.

Step 4: Diagnostic and Report Tool (DART)

Cisco provides a tool for troubleshooting AnyConnect connections: Diagnostics and Reporting Tool (DART). DART is a tool built in to AnyConnect that provides troubleshooting information regarding installation, configuration, and environmental problems. DART collects logs, diagnostics, and status information from AnyConnect and outputs them into a .zip file. DART is an AnyConnect module that must be deployed on a client workstation. The SVPN 300-730 exam does not cover detailed DART troubleshooting, but we mention DART because it can be useful for real-world deployments, and we encourage you to use it at this point of the troubleshooting process. You can use DART instead of running a long list of general diagnostic commands, and its results provide a general diagnostic view of AnyConnect.

Step 5: Diagnostic Commands

It is very possible a VPN will seem to be working correctly, but issues will come up such as packets being dropped, resources being blocked, or other problems that we have covered in this section. If you work through the previous steps and find that everything looks okay, but a problem still exists, you should try running a few diagnostic commands and reviewing the details. The SVPN 300-730 exam may show you output of a general diagnostic command and expect you to identify the problem in the output.

A very important general diagnostic command is the show vpn-sessiondb detail anyconnect command. This command shows you what resources are being used by AnyConnect, who the user is, counters for successful packets, and details about the VPN tunnel being used. Example 10-1 shows an example of running this diagnostic command.

Example 10-1 show vpn-sessiondb detail anyconnect Example

vpn-asa1# show vpn-sessiondb detail anyconnect
 
Session Type: AnyConnect Detailed
 
Username     : vpnuser                Index        : 9
Assigned IP  : 172.16.30.1            Public IP    : 198.18.100.10
Protocol     : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License      : AnyConnect Premium
Encryption   : AnyConnect-Parent: (1)none  SSL-Tunnel: (1)AES-GCM-256  DTLS-Tunnel: (1)AES-GCM-256
Hashing      : AnyConnect-Parent: (1)none  SSL-Tunnel: (1)SHA384  DTLS-Tunnel: (1)SHA384
Bytes Tx     : 15498                  Bytes Rx     : 12934
Pkts Tx      : 12                     Pkts Rx      : 48
Pkts Tx Drop : 0                      Pkts Rx Drop : 0
Group Policy : EMPLOYEE_GROUP         Tunnel Group : EMPLOYEE_CONNECTION
Login Time   : 04:31:57 UTC Mon Feb 22 2021
Duration     : 0h:01m:07s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : 6464018f00009000603333bd
Security Grp : none
 
AnyConnect-Parent Tunnels: 1
SSL-Tunnel Tunnels: 1
DTLS-Tunnel Tunnels: 1
 
AnyConnect-Parent:
  Tunnel ID    : 9.1
  Public IP    : 198.18.100.10
  Encryption   : none                   Hashing      : none
  TCP Src Port : 49870                  TCP Dst Port : 443
  Auth Mode    : userPassword
  Idle Time Out: 30 Minutes             Idle TO Left : 28 Minutes
  Client OS    : win
  Client OS Ver: 10.0.17763
  Client Type  : AnyConnect
  Client Ver   : Cisco AnyConnect VPN Agent for Windows 4.8.03052
  Bytes Tx     : 7749                   Bytes Rx     : 0
  Pkts Tx      : 6                      Pkts Rx      : 0
  Pkts Tx Drop : 0                      Pkts Rx Drop : 0
 
SSL-Tunnel:
  Tunnel ID    : 9.2
  Assigned IP  : 172.16.30.1            Public IP    : 198.18.100.10
  Encryption   : AES-GCM-256            Hashing      : SHA384
  Ciphersuite  : ECDHE-RSA-AES256-GCM-SHA384
  Encapsulation: TLSv1.2                TCP Src Port : 49874
  TCP Dst Port : 443                    Auth Mode    : userPassword
  Idle Time Out: 30 Minutes             Idle TO Left : 28 Minutes
  Client OS    : Windows
  Client Type  : SSL VPN Client
  Client Ver   : Cisco AnyConnect VPN Agent for Windows 4.8.03052
  Bytes Tx     : 7749                   Bytes Rx     : 10112
  Pkts Tx      : 6                      Pkts Rx      : 36
  Pkts Tx Drop : 0                      Pkts Rx Drop : 0
 
DTLS-Tunnel:
  Tunnel ID    : 9.3
  Assigned IP  : 172.16.30.1            Public IP    : 198.18.100.10
  Encryption   : AES-GCM-256            Hashing      : SHA384
  Ciphersuite  : ECDHE-ECDSA-AES256-GCM-SHA384
  Encapsulation: DTLSv1.2               UDP Src Port : 49771
  UDP Dst Port : 443                    Auth Mode    : userPassword
  Idle Time Out: 30 Minutes             Idle TO Left : 29 Minutes
  Client OS    : Windows
  Client Type  : DTLS VPN Client
  Client Ver   : Cisco AnyConnect VPN Agent for Windows 4.8.03052
  Bytes Tx     : 0                      Bytes Rx     : 2822
  Pkts Tx      : 0                      Pkts Rx      : 12
  Pkts Tx Drop : 0                      Pkts Rx Drop : 0

A more generic and less detailed version of the previous diagnostic command is the show vpn-sessiondb command. It is useful for verifying whether a VPN is running, how many clients are using the VPN, and load numbers. You don’t get as much detail from this command, but it is useful for getting a quick look at the VPN status. Example 10-2 shows an example of running the show vpn-sessiondb command.

Example 10-2 show vpn-sessiondb Example

vpn-asa1# show vpn-sessiondb
---------------------------------------------------------------------------
VPN Session Summary
---------------------------------------------------------------------------
                               Active : Cumulative : Peak Concur : Inactive
                             ----------------------------------------------
AnyConnect Client            :      1 :          2 :           1 :        0
  SSL/TLS/DTLS               :      1 :          2 :           1 :        0
Clientless VPN               :      0 :          3 :           1
  Browser                    :      0 :          3 :           1
---------------------------------------------------------------------------
Total Active and Inactive    :      1             Total Cumulative :      5
Device Total VPN Capacity    :    750
Device Load                  :     0%
---------------------------------------------------------------------------
 
---------------------------------------------------------------------------
Tunnels Summary
---------------------------------------------------------------------------
                               Active : Cumulative : Peak Concurrent
                             ----------------------------------------------
Clientless                   :      0 :          3 :               1
AnyConnect-Parent            :      1 :          2 :               1
SSL-Tunnel                   :      1 :          2 :               1
DTLS-Tunnel                  :      1 :          2 :               1
---------------------------------------------------------------------------
Totals                       :      3 :          9
---------------------------------------------------------------------------

Additional diagnostic commands to consider include the following:

• show version: Checks what type of code is running and shows the system status

• show cpu usage: Validates how the system is performing

• debug webvpn: Starts running general debugging for the VPN traffic

Step 6: Application

The final troubleshooting topic is verifying whether an issue is related to a specific application, such as an ASA plug-in or the user’s browser. Application troubleshooting mirrors client and clientless VPN deployment troubleshooting, and you use the same troubleshooting steps as described earlier in this chapter. As a brief reminder, you first research the application that is causing the problem and validate that it is supported by Cisco. Next, you validate connectivity between the ASA and the application as well as between the user’s browser and the application. Finally, you attempt to access the application from the user’s site and troubleshoot the user’s browser, including clearing the browser cache. Application troubleshooting is not likely to be on the SVPN 300-730 exam, but you might have to deal with it while supporting real-world ASA VPN deployments.

Troubleshooting AnyConnect IKEv2 VPNs on the ASA

The next troubleshooting focus area you must be familiar with to properly prepare for the Cisco SVPN 300-730 exam is troubleshooting AnyConnect IKEv2 VPNs on an ASA. As in the previous sections, in this section we provide a process you can step through to narrow down the possible issues. In this section and the next, we focus on AnyConnect IKEv2 VPNs both when using a Cisco ASA and when leveraging a Cisco router as the VPN headend. The focus is on the VPN tunnel and a few common host-side error messages; other troubleshooting steps, including testing connectivity and login problems, are covered earlier in this chapter. We mention those steps to keep the troubleshooting process complete, but we leave out the details to avoid repetition.

Step 0: Prepare

As with the other troubleshooting sections, the first part of troubleshooting any technology deployment is understanding what components are involved—what we call step zero. Troubleshooting AnyConnect IKEv2 VPNs means looking at an ASA that is acting as the VPN concentrator with AnyConnect playing the role of VPN client and the IKEv2 protocol being used for the VPN, as shown in Figure 10-25. It is highly recommended that you keep this simple architecture in your mind as you approach AnyConnect IKEv2 questions on the exam so you can quickly weed out any wrong answers referencing technology or configurations that aren’t being used.

In this section, as in the earlier sections of this chapter, we look at troubleshooting in different section. First, we look at connectivity to the ASA. Steps for troubleshooting connectivity are covered earlier in this chapter and are not repeated here. If the ASA can’t be reached, you know the issue is likely network or certificate related, and the focus can be on fixing these types of problems. If connectivity works but you can’t log in to the VPN after making a connection, troubleshooting should focus on login challenges, which is also covered earlier in this chapter.

Once you have validated that connectivity and login issues do not exist, you focus on the VPN connection status, which is the focus of this section and the next. Our focus on troubleshooting the VPN connection will be limited to a handful of specific commands you can use to validate the VPN status for a IKEv2 VPN deployment. The details provided from using these commands should give you enough information to troubleshoot the VPN connection status. You need to be familiar with all aspects of using these commands for the SVPN 300-730 exam, including understand what each command does, what type of output it shows, and how the output relates to troubleshooting.

Finally, troubleshooting shifts to the client side if you do not have access to the ASA command line, if you want to test VPN problems from the client side, or if the exam asks questions regarding host-based troubleshooting. We touched on general host troubleshooting, including host browser settings and potential browser problems, earlier in this chapter, and this section covers just a few common host errors found specifically in AnyConnect IKEv2 deployments.

Troubleshooting AnyConnect IKEv2 VPNs involves the following steps:

Step 1. Connectivity: Confirm that the system looking to connect to the VPN tunnel can reach the ASA or router.

• Can you ping between systems?

• Validate certificate configuration.

Step 2. Login: Troubleshoot any login issues, including ensuring that the proper user group is selected and that the user is able to pass all authentication and authorization steps.

• Tunnel group selection

• Authentication

• Authorization

• Group policy selection

Step 3. IKEv2 status validation: Run commands to validate the status of the VPN.

• Run the command show vpn-sessiondb detail anyconnect

• Run the command show crypto ikev2 sa

• Run the command show crypto ikev2 sa detail

• Run the command debug crypto ikev2

Step 4. Host: Troubleshoot issues related to accessing the application.

• Check for an invalid host entry.

• View the AnyConnect agent by using the DART tool or general agent logs to confirm that there isn’t a host side issue.

Let’s walk through these steps in more detail, starting with troubleshooting connectivity and login issues.

Steps 1 and 2: Connectivity and Login to the VPN Concentrator

All troubleshooting steps start off by validating connectivity to the VPN concentrator (ASA or router). This is the most common place for an issue to occur. Issues could occur during the initial deployment but more commonly come up later, while the VPN solution is in operation. Network changes, such as a firewall rule change that disables Internet access to the ASA, commonly break working VPN deployments. Therefore, troubleshooting should focus on performing all standard network validation efforts, such as pinging each device involved to validate that the devices are reachable (device-to-ASA and vice versa) and confirming that a firewall or other technology along the path of the connection hasn’t been changed. We cover how to troubleshoot connectivity earlier in this chapter, and the troubleshooting steps are the same for any type of VPN setup.

Once connectivity is validated, you need to confirm that users can log in. This includes ensuring that the connection/group is properly configured, the user is properly authenticated, and the user is authorized for the privileges associated with the assigned group. We cover troubleshooting ASA login issues earlier in this chapter, and the troubleshooting steps are the same, regardless of the type of ASA VPN being deployed.

If connectivity and login testing confirm that there are no issues, the next step for troubleshooting an IKEv2 VPN is to troubleshoot the VPN connection. This brings us to the third step in troubleshooting a IKEv2 VPN.

Step 3: VPN Status Validation

If the connectivity and login is confirmed to be working, the next step for troubleshooting an IKEv2 VPN is validating that the VPN is working from the VPN concentrator viewpoint. SVPN 300-730 exam troubleshooting questions will mostly provide snippets of either debugging code or show commands as your only view into troubleshooting a connectivity problem. You will be expected to review the output of a command and determine the most likely cause of the issue. It is very unlikely that the exam will test you on something as simple as a ping showing that something is unavailable, so make sure to be familiar with all steps that occur during an IKEv2 VPN connection, as explained in Chapter 8. You might, however, find that a ping is successful but the VPN does not works due to the UDP and TCP ports for the VPN being filtered. For SSLVPN, TCP and UDP port 443 must be available. For IKE, UDP port 500 and UDP port 4500 are used. Questions around required ports are fair game on the SVPN 300-730 exam.

This troubleshooting section focuses on validating different aspects of an IKEv2-based VPN tunnel using the ASA command line. For the exam and real-world deployments, the most common ASA commands that you need to be familiar with for IKEv2 troubleshooting with AnyConnect are as follows:

• show vpn-sessiondb detail anyconnect: The show vpn-sessiondb command is used to view summary information about the VPN session. Adding detail anyconnect provides even more details about AnyConnect sessions.

• show crypto ikev2 sa: This command displays the state of the Phase 1 tunnel

• show crypto ikev2 sa detail: This command displays more details about the state of the Phase 1 tunnel

• show crypto ipsec sa: This command displays details about the state of the Phase 2 tunnel

• debug crypto ikev2 255: This command enables debugging, which provides details on current crypto IKEv2 activity

Each of these commands can be used to validate the status of a VPN session. The following sections look at them in more detail.

Command 1: show vpn-sessiondb detail anyconnect

When you run the command show vpn-sessiondb detail anyconnect on the ASA command line, you need to look for an established VPN connection, validate the user and network information, validate the protocol used, check the group policy and connection profile information, check the authentication method used, and check NAT information. Example 10-3 shows a healthy VPN connection.

For the SVPN 300-730 exam and in the real world, you need to know how to validate proper operation of any of the details seen in this output. The exam will expect you to be able to interpretate this output. Notice that you can validate the username, IP address information, protocol, license being used, type of encryption (in this example, AES-256), hashing (in this example, SHA-1), group policy, connection profile, login time, and many other details. (We cover how to set up an ASA VPN using IKEv2 in Chapter 9.) Be familiar with all data points shown in Example 10-3 and know how to flag errors in configuration compared to expected VPN configurations as you are troubleshooting.

Example 10-3 show vpn-sessiondb detail anyconnect Example

vpn-asa1# show vpn-sessiondb detail anyconnect
 
Session Type: AnyConnect Detailed
 
Username     : vpnuser                Index        : 11
Assigned IP  : 172.16.30.1            Public IP    : 198.18.100.10
Protocol     : IKEv2 IPsecOverNatT AnyConnect-Parent
License      : AnyConnect Premium
Encryption   : IKEv2: (1)AES256  IPsecOverNatT: (1)AES256  AnyConnect-Parent: (1)none
Hashing      : IKEv2: (1)SHA1  IPsecOverNatT: (1)SHA1  AnyConnect-Parent: (1)none
Bytes Tx     : 0                      Bytes Rx     : 12251
Pkts Tx      : 0                      Pkts Rx      : 45
Pkts Tx Drop : 0                      Pkts Rx Drop : 0
Group Policy : EMPLOYEE_GROUP         Tunnel Group : EMPLOYEE_CONNECTION
Login Time   : 04:46:27 UTC Mon Feb 22 2021
Duration     : 0h:00m:11s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : 6464018f0000b00060333723
Security Grp : none
 
IKEv2 Tunnels: 1
IPsecOverNatT Tunnels: 1
AnyConnect-Parent Tunnels: 1
 
AnyConnect-Parent:
  Tunnel ID    : 11.1
  Public IP    : 198.18.100.10
  Encryption   : none                   Hashing      : none
  Auth Mode    : userPassword
  Idle Time Out: 30 Minutes             Idle TO Left : 29 Minutes
  Client OS    : win
  Client OS Ver: 10.0.17763
  Client Type  : AnyConnect
  Client Ver   : 4.8.03052
 
IKEv2:
  Tunnel ID    : 11.2
  UDP Src Port : 59697                  UDP Dst Port : 4500
  Rem Auth Mode: userPassword
  Loc Auth Mode: rsaCertificate
  Encryption   : AES256                 Hashing      : SHA1
  Rekey Int (T): 86400 Seconds          Rekey Left(T): 86387 Seconds
  PRF          : SHA1                   D/H Group    : 2
  Filter Name  :
  Client OS    : Windows                Client Type  : AnyConnect
 
IPsecOverNatT:
  Tunnel ID    : 11.3
  Local Addr   : 0.0.0.0/0.0.0.0/0/0
  Remote Addr  : 172.16.30.1/255.255.255.255/0/0
  Encryption   : AES256                 Hashing      : SHA1
  Encapsulation: Tunnel
  Rekey Int (T): 28800 Seconds          Rekey Left(T): 28787 Seconds
  Idle Time Out: 30 Minutes             Idle TO Left : 29 Minutes
  Bytes Tx     : 0                      Bytes Rx     : 12416
  Pkts Tx      : 0                      Pkts Rx      : 46

Command 2: show crypto ikev2 sa

The next command on our list is the show crypto ikev2 sa command. When you execute this command, your goal is to validate the status of Phase 1 by looking for the UP-ACTIVE status. You can also validate the IP address ranges being used and details regarding the session. In a real-world deployment, you will likely use the detailed version of this command. However, the exam may only provide the standard show version, as shown in Example 10-4.

Example 10-4 show crypto ikev2 sa Example

vpn-asa1# show crypto ikev2 sa
 
IKEv2 SAs:
 
Session-id:1, Status:UP-ACTIVE, IKE count:1, CHILD count:1
 
Tunnel-id Local                                               Remote
Status         Role
222048265 203.0.113.30/4500                                   198.18.100.10/59697
READY    RESPONDER
      Encr: AES-CBC, keysize: 256, Hash: SHA96, DH Grp:2, Auth sign: RSA, Auth verify: EAP
      Life/Active Time: 86400/101 sec
Child sa: local selector  0.0.0.0/0 - 255.255.255.255/65535
          remote selector 172.16.30.1/0 - 172.16.30.1/65535
          ESP spi in/out: 0x873fc6b5/0xaa72493d

Command 3: show crypto ikev2 sa detail

The show crypto ikev2 sa detail command adds detail to the previous command, allowing for more details to be included with the show output. In practice, you should use the detail version of this command, which gives you data counts, SPI details, child and parent SA status, and other details. Example 10-5 shows sample output of using the show crypto ikev2 sa detail command. We highly recommend being familiar with both versions of this command for the SVPN 300-730 exam.

Example 10-5 show crypto ikev2 sa detail Example

vpn-asa1# show crypto ikev2 sa detail
 
IKEv2 SAs:
 
Session-id:1, Status:UP-ACTIVE, IKE count:1, CHILD count:1
 
Tunnel-id Local                                               Remote
Status         Role
222048265 203.0.113.30/4500                                   198.18.100.10/59697
READY    RESPONDER
      Encr: AES-CBC, keysize: 256, Hash: SHA96, DH Grp:2, Auth sign: RSA, Auth verify: EAP
      Life/Active Time: 86400/147 sec
      Session-id: 1
      Status Description: Negotiation done
      Local spi: 61C61EC6967F5ABF       Remote spi: D7B7EBC81348D7D1
      Local id: cn=vpn-asa1.example.com
      Remote id: *$AnyConnectClient$*
      Local req mess id: 0              Remote req mess id: 10
      Local next mess id: 0             Remote next mess id: 10
      Local req queued: 0               Remote req queued: 10
      Local window: 1                   Remote window: 1
      DPD configured for 30 seconds, retry 2
      NAT-T is detected  outside
      Assigned host addr: 172.16.30.1
      IKEv2 Fragmentation Configured MTU: 576 bytes, Overhead: 28 bytes, Effective MTU: 548 bytes
Child sa: local selector  0.0.0.0/0 - 255.255.255.255/65535
          remote selector 172.16.30.1/0 - 172.16.30.1/65535
          ESP spi in/out: 0x873fc6b5/0xaa72493d
          AH spi in/out: 0x0/0x0
          CPI in/out: 0x0/0x0
          Encr: AES-CBC, keysize: 256, esp_hmac: SHA96
          ah_hmac: None, comp: IPCOMP_NONE, mode tunnel
Parent SA Extended Status:
      Delete in progress: FALSE
      Marked for delete: FALSE

Command 4: show crypto ipsec sa

The show crypto ipsec sa command is important for troubleshooting Phase 2 and provides important details regarding the status of the VPN connection. The SVPN 300-730 exam may give you parts of this show output and ask you why an issue is occurring. This command includes various counters, including counters that are triggered when a failure occurs. The exam might ask you why a problem is occurring when it shows output with #pre-frag failures increasing in count. It might also ask you to confirm details in the output of this command, such as specifics in the outbound esp sas section.

A very common issue that can be identified with this command is traffic being blocked in one direction. In Example 10-6, #pkts encaps is 47, and #pkts decaps is 0. Such output can occur for two reasons: Either no traffic is being sent by the other side of the connection or traffic is being sent by the other side of the connection but is blocked along the path.

Example 10-6 show crypto ipsec sa Example

vpn-asa1# show crypto ipsec sa
interface: OUTSIDE
    Crypto map tag: SYSTEM_DEFAULT_CRYPTO_MAP, seq num: 65535, local addr: 203.0.113.30
 
      local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
      remote ident (addr/mask/prot/port): (172.16.30.1/255.255.255.255/0/0)
      current_peer: 198.18.100.10, username: vpnuser
      dynamic allocated peer ip: 172.16.30.1
      dynamic allocated peer ip(ipv6): 0.0.0.0
 
      #pkts encaps: 47, #pkts encrypt: 47, #pkts digest: 47
      #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #TFC rcvd: 0, #TFC sent: 0
      #Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
      #send errors: 0, #recv errors: 0
 
      local crypto endpt.: 203.0.113.30/4500, remote crypto endpt.: 198.18.100.10/59697
      path mtu 1472, ipsec overhead 66(52), override mtu 1406, media mtu 1500
      PMTU time remaining (sec): 0, DF policy: copy-df
      ICMP error validation: disabled, TFC packets: disabled
      current outbound spi: AA72493D
      current inbound spi : 873FC6B5
 
    inbound esp sas:
      spi: 0x873FC6B5 (2269103797)
         SA State: active
         transform: esp-aes-256 esp-sha-hmac no compression
         in use settings ={RA, Tunnel,  NAT-T-Encaps, IKEv2, }
         slot: 0, conn_id: 11, crypto-map: SYSTEM_DEFAULT_CRYPTO_MAP
         sa timing: remaining key lifetime (sec): 28611
         IV size: 16 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x0000FFFF 0xFFFFFFFF
    outbound esp sas:
      spi: 0xAA72493D (2859616573)
         SA State: active
         transform: esp-aes-256 esp-sha-hmac no compression
         in use settings ={RA, Tunnel,  NAT-T-Encaps, IKEv2, }
         slot: 0, conn_id: 11, crypto-map: SYSTEM_DEFAULT_CRYPTO_MAP
         sa timing: remaining key lifetime (sec): 28611
         IV size: 16 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

Command 5: debug crypto ikev2 255

The debug crypto ikev2 255 command enables debugging. Logging messages start to appear when you attempt the VPN connection and often include log messages both before and after failure occurs. To disable debugging, you use the command no debug all. Unlike with routers, though, the ASA stops debugging when you log off.

Step 4: Host Troubleshooting

Another focus area for troubleshooting is the host running AnyConnect representing the system attempting to connect to the VPN concentrator (the ASA). It is likely that issues will first be discovered on hosts. That is, end users may experience problems and escalate the issue for administrators to troubleshoot. The focus of this section is on evaluating a few common error messages that can come up when you run an IKEv2 VPN deployment.

Chapter 9 describes how to view details about remote access VPN status using tools available in AnyConnect. Earlier in this chapter, you learned about common general host problems that occur due to browser problems, which would be identical for troubleshooting any type of VPN from the host viewpoint. Make sure you are familiar with validating VPN status from the host level as the SVPN 300-730 exam could include screenshots of output from AnyConnect and ask you to validate what is being shown.

The following sections look at two error messages your end users may experience when things are not working with an IKEv2 VPN set up on a Cisco ASA.

Invalid Host Entry

One possible error message a user may experience is the AnyConnect message “Invalid host entry. Please re-enter.” This error occurs when the user group name in the XML client profile does not match the name of the connection profile on the ASA. The user group name and connection profile name must be the same. Otherwise, the “Invalid host entry. Please re-enter.” error message appears on the AnyConnect client, as shown in Figure 10-26.

Chapter 9 uses the user group EMPLOYEES for a configuration example. The user group EMPLOYEES in Figure 10-26 is the reason for the error that is being displayed. Chapter 9 calls the connection profile EMPLOYEE_CONNECTION, which is different from EMPLOYEES. Unlike with SSLVPN connections, where the user group does not have to match the connection profile name, the user group for IKEv2 connections must match the connection profile name exactly. You can validate this configuration in the AnyConnect client profiler editor by going to the Server List screen, as shown in Figure 10-27.

There are other issues that could come up, and we recommend leveraging DART to view the log details. The SVPN 300-730 exam will likely use DART output od to test you on troubleshooting AnyConnect-related issues.

Troubleshooting AnyConnect IKEv2 VPNs on Routers

The final troubleshooting topic to cover is AnyConnect IKEv2 VPNs on routers. Up until this point in the chapter, we have focused on IKEv2 troubleshooting commands from the ASA command line. The Cisco SVPN 300-730 exam also expects you know router-based VPN troubleshooting, which is the focus of this section. The basic components involved with an AnyConnect IKEv2-based VPN on a router are the AnyConnect client and a router acting as the VPN concentrator. Figure 10-28 shows an example of the configuration used for this section’s troubleshooting examples.

Many of the steps for troubleshooting a router-based VPN are identical to the steps described for using an ASA. You should test connectivity to each device, validate the certificates, confirm that the login process works, test the IKEv2 status by using the router command line, and check the status of AnyConnect on the host.

Troubleshooting router-based AnyConnect IKEv2 VPNs involves the following steps:

Step 1. Connectivity: Confirm that the system looking to connect to the VPN tunnel can reach the router.

• Can you ping between systems?

• Validate certificate configuration.

Step 2. Login: Troubleshoot any login issues, including confirming that the proper user group is selected and the user is able to pass all authentication and authorization steps.

• Group selection

• Authentication

• Authorization

Step 3. IKEv2 status validation: Run commands to validate the status of the VPN.

• Run the command show crypto ipsec sa detail

• Run the command show crypto session detail

• Run the command debug aaa commands

• Run the command debug crypto ikev2

Step 4. Hosts: Troubleshoot issues related to accessing the application.

• Check for invalid host entry.

• View the AnyConnect agent by using DART or general agent logs to confirm that there isn’t a host side issue.

Many of these steps are exactly the same as the ones covered earlier in this chapter.

Steps 1 and 2: Connectivity and Login to the Router

By now you know that any troubleshooting starts with testing connectivity. You must first confirm that everything is reachable by using ping to ensure that a security tool is not preventing communication or something else is not causing a disruption in connection. Next, you need to make sure the certificates are valid. After testing connectivity, you need to confirm that the login process functions properly. If connectivity testing confirms that there are no issues and the login process works, the next step for troubleshooting an IKEv2 VPN is to troubleshoot the VPN connection. Let’s look at how this works on a Cisco router.

Step 3: VPN Status Validation

If connectivity and login troubleshooting show that things are working, the next step you should take is validating the status of the VPN. This section focuses on validating different aspects of an IKEv2-based VPN tunnel on a Cisco router.

For the SVPN 300-730 exam, you should be familiar with the following router commands:

• show crypto ipsec sa detail: This command displays details about the state of the Phase 2 tunnel.

• show crypto session detail: This command provides details on the current status of the crypto session.

• debug aaa commands: These commands are used to validate the authentication and authorization lists that are used as well as troubleshooting any AAA-related issues.

• debug crypto ikev2: This command enables debugging for crypto IKEv2 to provide live details about the IKEv2 status.

Notice that a few of these commands are similar or identical to the ones just covered for troubleshooting an ASA IKEv2 VPN. The output will be different on a router, but the general details displayed will provide similar data points that you will need in order to validate the status of the VPN.

Let’s look more closely at these commands.

Command 1: show crypto ipsec sa detail

The show crypto ipsec sa detail command that you have already seen for troubleshooting IKEv2-based VPNs using an ASA is also used with routers. You use it to validate the status of the connection, determine which devices are involved, learn what is enabled or disabled, and get SPI info and other details. Example 10-7 shows an example of issuing the show crypto ipsec sa detail command on a router; as you can see, this is very similar to running this command at the CLI of a Cisco ASA.

Example 10-7 show crypto ikev2 sa detailed Example

VPN-ROUTER# show crypto ikev2 sa detailed
 IPv4 Crypto IKEv2  SA
 
Tunnel-id Local                 Remote                fvrf/ivrf            Status
1         203.0.113.20/4500     198.18.100.10/63454   none/none            READY
      Encr: AES-CBC, keysize: 256, PRF: SHA512, Hash: SHA512, DH Grp:21, Auth sign: RSA, Auth 
verify: AnyConnect-EAP
      Life/Active Time: 86400/48 sec
      CE id: 1016, Session-id: 8
      Status Description: Negotiation done
      Local spi: 24C27AD8AF48D3E9       Remote spi: 49A438F8575181B2
      Local id: 203.0.113.20
      Remote id: *$AnyConnectClient$*
      Remote EAP id: vpnuser
      Local req msg id:  0              Remote req msg id:  7
      Local next msg id: 0              Remote next msg id: 7
      Local req queued:  0              Remote req queued:  7
      Local window:      5              Remote window:      1
      DPD configured for 0 seconds, retry 0
      Fragmentation not  configured.
      Dynamic Route Update: disabled
      Extended Authentication not configured.
      NAT-T is detected  outside
      Cisco Trust Security SGT is disabled
      Assigned host addr: 172.16.2.8
      Initiator of SA : No
 
 IPv6 Crypto IKEv2  SA

Command 2: show crypto session detail

As you have already seen for troubleshooting on an ASA, the crypto session detail command is used to validate the details of the crypto session on a router. The SVPN 300-730 exam may use the abbreviated version of this command, but real-world troubleshooting typically requires the detail version. Details include the profile being used, uptime, status, IP addresses for each side, and other details, as shown in Example 10-8.

Example 10-8 show crypto session detail Example

VPN-ROUTER# show crypto session detail
Crypto session current status
 
Code: C - IKE Configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal, T - cTCP encapsulation
X - IKE Extended Authentication, F - IKE Fragmentation
R - IKE Auto Reconnect, U - IKE Dynamic Route Update
S - SIP VPN
 
Interface: Virtual-Access1
Profile: LOCAL_USER_IKEV2_PROFILE
Uptime: 00:00:11
Session status: UP-ACTIVE
Peer: 198.18.100.10 port 63454 fvrf: (none) ivrf: (none)
      Phase1_id: *$AnyConnectClient$*
      Desc: (none)
  Session ID: 32
  IKEv2 SA: local 203.0.113.20/4500 remote 198.18.100.10/63454 Active
          Capabilities:N connid:1 lifetime:23:59:49
  IPSEC FLOW: permit ip 0.0.0.0/0.0.0.0 host 172.16.2.8
        Active SAs: 2, origin: crypto map
        Inbound:  #pkts dec'ed 48 drop 0 life (KB/Sec) 4607985/3589
        Outbound: #pkts enc'ed 0 drop 0 life (KB/Sec) 4608000/3589

Command 3: debug aaa

You use general AAA debug commands to determine whether authentication and authorization are functioning properly. As with any other debug command, you must first enable debugging before you attempt to perform an activity or wait for the activity to occur before data will show. Example 10-9 shows an example of enabling debugging for authorization and authentication on a router. The same commands can be used on an ASA, and with either an ASA or a router, the goal is to validate the correct authentication and authorization groups. In Example 10-9, LOCAL_USER_AUTHC is the authentication list being used, and LOCAL_GROUP-AUTHZ is the authorization list being used. The SVPN 300-730 exam may show you debug aaa output and ask you to validate which list is being used. Understanding debug aaa output can help you easily select the correct response.

Example 10-9 debug aaa Example

VPN-ROUTER# debug aaa authentication
AAA Authentication debugging is on
VPN-ROUTER# debug aaa authorization
AAA Authorization debugging is on
*Feb 23 03:44:58.203: AAA/BIND(0000001C): Bind i/f
*Feb 23 03:44:58.204: AAA/AUTHEN/LOGIN (0000001C): Pick method list 'LOCAL_USER_AUTHC'
*Feb 23 03:44:58.211: AAA/BIND(0000001D): Bind i/f
*Feb 23 03:44:58.211: AAA/AUTHOR (0x1D): Pick method list 'LOCAL_GROUP_AUTHZ'
*Feb 23 03:44:58.213: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access1, changed 
state to down
*Feb 23 03:44:58.246: %SYS-5-CONFIG_P: Configured programmatically by process Crypto INT from 
console as console
*Feb 23 03:44:58.286: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access1, changed 
state to up

Summary

Troubleshooting a VPN requires a strong understanding of how VPN technology works. If you don’t feel confident in the deployment of any of the four VPN designs covered in this chapter, you need to go back to the chapter that covers how the technology works and master how to build the VPN before you invest time understanding associated troubleshooting concepts. Consider studying troubleshooting as a way to validate what you already know about a particular VPN technology.

The general steps for troubleshooting VPN technology covered in this chapter can be broken into a number of focus areas. Before taking any action, we recommended a taking a step zero to understand the technology you are working with. After you have researched what you will be working with, you can begin troubleshooting connectivity concepts to ensure that communication is functioning. This includes troubleshooting certificate and DNS problems as well as using ping to confirm connectivity. Next, you can look at how users log in to the VPN. When you have eliminated connectivity and login problems as the cause of a VPN issue, you can move to troubleshooting VPN services. For SSL, this includes focusing on the WebVPN service, bookmarks, and applications; with IKEv2, troubleshooting includes running a handful of commands to view different aspects of the VPN status. Finally, troubleshooting should focus on the host, looking at problems specific to the user’s system. Problems could include the user’s browser or system not supporting different aspects of the VPN, and you would need to work with the user to address such problems.

Even though you have finished reading the topical chapters of this book, you need to do more to prepare for the SVPN 300-730 exam and master configuring, running, and troubleshooting VPN networks. Security is a journey, not a destination, and you cannot master it just by reading a book. Mastering a skill requires practice. We have included a bunch of sample questions you should work through not with the focus of memorizing, but with a focus on understanding why each question is being asked so you can challenge your knowledge of the topic being addressed. We also highly recommend working with VPN technology in a lab environment so you can better understand when to use the steps covered here as well as what the output of certain commands looks like. You could also use a search engine to find videos and screenshots of commands or walkthroughs of topics covered in this book.

This book provides coverage of the SVPN 300-730 exam topics, and you will be expected to know everything covered here, including different viewpoints, such as command-line code snippets, error messages, host VPN or administration viewpoints, and everything in between. In short, you should study the concepts in this book and leverage different resources to further review what you believe you need to master before you can feel confident going into the SVPN 300-730 exam.

We appreciate your investment in this resource and wish you the best of luck with the SVPN 300-730 exam as well as future work with VPN technology. Thank you from the writing team!

Reference

ASA Clientless VPN Troubleshooting. Retrieved from https://www.cisco.com/c/en/us/td/docs/security/asa/asa96/configuration/vpn/asa-96-vpn-config/webvpn-troubleshooting.html#ID-2276-00000005

Exam Preparation Tasks

As mentioned in the section “How to Use This Book” in the Introduction, you have a couple of choices for exam preparation: the exercises here, Chapter 11, “Final Preparation,” and the exam simulation questions in the Pearson Test Prep software online.

Review All Key Topics

Review the most important topics in the chapter, noted with the key topics icon in the outer margin of the page. Table 10-2 lists these key topics and the page number on which each is found.

Complete Tables and Lists from Memory

This chapter does not have any memory tables.

Define Key Term

Define the following key term from this chapter and check your answers in the glossary:

WebVPN

Use the Command Reference to Check Your Memory

Table 10-3 lists the important commands from this chapter. To test your memory, cover the right side of the table with a piece of paper, read the description on the left side, and see how much of the command you can remember.