NetScaler has extensive logging and tracing capabilities and includes some GUI-based tools that help you with issue analysis. To complement these, Citrix also provides external tools to support you with your NetScaler deployments and associated troubleshooting. A discussion of these various tools is the focus of this chapter. We will touch on these utilities in the following order:
nstrace
)showtechsupport
utilityNsconmsg is a logging process that runs nonstop on the NetScaler. It contains a snapshot of all the available counters taken once every 7 seconds. There are an excess of 10,000 counters each with specific conditions for specific features, which makes this such a formidable troubleshooting tool in identifying if a problem condition has been hit.
The logs produced by this utility are available in /var/nslog
. By default, they roll over once every two days or on reaching a size of 300 MB whichever is first. The file newlog
will always be latest (current log file). They can go up to 99 files after which they restart from 0.
nsconmsg
is used from the shell prompt with the following syntax:
nsconmsg –K <path> –g <counter_name> -d <display_option>
The commonly used options are as follows:
-K
: This is used to specify the name of the log file; if not used, current values of counters are shown.-g
: This is used to grep for the values of specific counters, making nsconmsg
more usable. For example, use –g ssl
to see all available SSL counters before narrowing down to specific ones.-d
: This is used to specify the type of operation. The common ones are:setime
: This displays the start and end time of the newslog
file.current
: This displays the current value of each counter, on a 7-second basis.event
: This displays all events during that log period. It is useful for looking for any service or monitor failures and HA-related events.oldconmsg
: This is used for load balancing and content switching; it allows you to see a variety of information such as the number of requests received by each service, number of active transaction, and how the surge queue looks at each 7-second interval.stats
: This displays the current statistics. It is useful to see whether a particular counter (and thus a specific error condition) has ever been incremented.statswt0
: This displays only counters that incremented during the current log.memstats
: This displays memory-related statistics.consmsg
: This displays console messages.Here are some useful examples:
setime
to display the time covered by the logs. If you know the time of the issue, this command will help zero in on the newnslog
file you need for analysis:nsconmsg -K /var/nslog/newnslog.x.tar.gz -d setime
current
to find out the rate at which a counter is incrementing:nsconmsg -K /var/nslog/newnslog -g <counter name> -d current
In the following example, I am trying to identify whether a rate limit is being hit and at which point packets will be dropped. The pattern I would grep
for this would -g _rl_
.
The easiest way to arrive at the counters you need is to Google search for various NetScaler counters. For example, to find the rate limiting counter I used in the screenshot, I looked for all NetScaler NIC-related counters and found the article CTX132772. Similar articles exist for all features. You can then use the –g
option to see whether the counter is being hit.
Some of the common patterns I use are -g ssl_err
, -g http_err
, -g tcp_err
, -g nic_err
, and –g csw_err
.
event
to look at event-related data:nsconmsg -K /var/nslog/newnslog -d event
This command is very useful to see if and when a particular failure event such as a HA failover (by grepping for the word node
) has occurred.
Another important use case for event data is checking for services and vServer flaps.
oldconmsg
to obtain traffic management data. Load balancing as well as other feature level info such as for content switching, SSL, or monitors can be extracted using the -d oldconmsg
switch by specifying the -s feature-specific
switch. For example:nsconmsg -K /var/nslog/newnslog -s ConLB=1 -d oldconmsg nsconmsg -K /var/nslog/newnslog -s ConCSW=1 -d oldconmsg
The output of these commands were discussed in detail in Chapter 2, Traffic Management Features.
18.221.25.217