IIS Logging

IIS logs all web site activity to SharePoint. While not necessarily the primary place to examine for errors or performance, it can provide an indication of issues users are running into, including missing assets or server errors, such as HTTP 500 errors.

As IIS logs are plain text files and parsing them can be difficult with text editors like Notepad, Log Parser and Log Parser Studio from Microsoft makes finding specific types of log entries significantly easier.

You may consider adding additional fields to log for each request. This can be done in IIS on a server or IIS web site level under Logging in the feature pane.

With this output, we can see there are 3 days where users received an HTTP 404 when requesting a resource. We know from this example that SharePoint does not include certain files, such as favicon.ico by default and can ignore these particular missing files.

And finally, based on the correlation ID, using a tool such as ULS Viewer, we can further examine the errors generated. In the case of the preceding error, it was due to a registry key that was attempted to be used even though it was marked for deletion. Resolving this error typically involves simply restarting the server as Windows deletes registry keys during the reboot process.

ULS Logging

ULS provides a valuable source of information about your SharePoint farm. This is the core logging mechanism of SharePoint and is often the first place a SharePoint Administrator will look for any SharePoint-related errors. By default, ULS logs are located in C:Program FilesCommon Filesmicrosoft sharedWeb Server Extensions16LOGS. ULS logs are in the format of ServerName-YYYYMMdd-hhmm.log, for example, CALSP01-20181114-0836.log.

The log location may also be found via Central Administration. Using Central Administration, navigate to Monitoring. Under Configure diagnostic logging, the Trace Log Path is where the ULS log is located, as shown in Figure 19-1.

Users may encounter errors from SharePoint, which provides them the date and time the error occurred, as well as the ULS Correlation ID. An example of one such error is seen in Figure 19-2.

Using this information, the Correlation ID and Date and Time, and using Ulsviewer, open the appropriate ULS log file. By using Ulsviewer, we can filter by the preceding Correlation ID, as shown in Figure 19-3, to see the end user’s request end to end.

The error may be identified within the list of entries once filtered. In this case, the error is generic, but the user had requested a Content Type that does not exist, seen in Figure 19-4.

The ULS log will display the date and time the log entry is from, the product (e.g., SharePoint, Project Server, PowerPivot, etc.), the Category (User Profiles, Search), the Event ID, the level (Unexpected are generally errors), Correlation ID, Message, the Request, and other information depending on the type of error.

Event IDs are used internally by Microsoft and the information of what message they’re associated with is not generally published.

It is possible that errors may not be caught using the default logging settings. For this, we need to increase the verbosity of logs. The verbosity settings are based on Areas. These settings can be modified via Central Administration under Monitoring, Configuring diagnostic logging.

This page will list the current verbosity level for each Area, as shown in Figure 19-5, as well as provide two drop-downs to adjust the verbosity between None to Verbose or allowing you to Reset to Default.

When using Verbose or VerboseEx trace levels, there may be a significant impact on farm performance. Because of this, you may want to run with these Trace Severities for a short period of time to reproduce a specific issue.

Once completed reproducing the issue, use Clear-SPLogLevel to reset all Areas back to their default Trace Severity.

ULS Viewer can be used for monitoring the live environment, as well. This is suitable when having a user reproduce a problem that does not necessarily surface an error, but will allow you to correlate the user’s action with one or more messages within the ULS log. The latest version of ULS Viewer is also able to monitor logs over the entire farm. By selecting the farm icon, represented by a tree node in the toolbar, you can enter one or more server names into the farm, then using a UNC path, Ulsviewer will allow you to see the server logs intermixed, real time. This is useful in scenarios where a user may call a service on a backend server, but you must trace the action of the user through the frontend to the backend.

Event Viewer

SharePoint records a limited amount of information to the Event Viewer, but the Event Viewer is more useful for service-specific and ASP.NET errors.

Generally, Windows Services that run SharePoint, such as the SharePoint Timer or SharePoint Administration service, will show any startup or unexpected stops in the System Event Log. For example, if the SharePoint Timer service unexpectedly stops, it will show an error in the System Event Log as seen in Figure 19-6.

The System Event Log is also useful for diagnosing Kerberos errors, along with any TLS/SSL errors that may occur.

The Application Event Log will show other more general SharePoint information, warnings, and error messages from a variety of sources. As an example, it will show when an IIS Application Pool has started, as shown in Figure 19-7.

SharePoint also logs data in a few Applications and Services event logs. In the Operational log for SharePoint Products, Shared, log entries typically consist of Incoming E-mail statistics, Usage and Trace Log status, such as when the log reached the retention limit based on space used or date as shown in Figure 19-8, and InfoPath Forms Services messages.

IIS Manager

The IIS Manager provides a limited amount of information on active requests in Application Pools. This information may be helpful for diagnosing the origins of long-running requests, for example, a large number of requests to a OneNote notebook residing on a SharePoint site.

Using IIS Manager, at the server level, go into “Worker Processes.” From here, as shown in Figure 19-9, it will show a limited amount of information about each worker process.

By right-clicking a Worker Process and selecting View Current Requests, we can identify running requests as shown in Figure 19-10.

Usage Logging

SharePoint Usage Logging logs a variety of information to the Usage database. This database can be directly queried either through the tables or through the built-in Views. For example, the RequestUsage View can provide information on how long a particular request took, how many CPU megacycle it consumed, Distributed Cache reads and how long those Distributed Cache reads took, among other statistics.

Usage Logging can be configured in Central Administration under Monitoring, Configure usage and health data collection. There are a number of scenarios to gather data on, but only gather those scenarios you believe will be important for farm diagnostics. Logging more than is required may lead to farm performance issues as the data is transferred from the SharePoint farm into the Usage database, along with the Usage database size growth.

The Usage database may be queried directly via SQL Server Management Studio. Microsoft provisions Views for many common scenarios one may be interested in, as shown in Figure 19-11, but you may also construct your own Views within the database if needed.

Querying the database is simple. As shown in Figure 19-12, construct your query of a View and select the columns you wish to display in the results, in the order you wish to display the results in. In this query, we are looking at the Administrative Actions View and select just the relevant columns that we’re interested in, then sorting by the time the log entry was created in the database, with the newest entries appearing first.

Central Administration Health Analyzer

The built-in SharePoint Health Analyzer is a set of rules that run periodically via the SharePoint Timer Service. These rules detect various issues, as shown in Figure 19-13, such as SharePoint Application Pools recycling, or databases with a large amount of free space, and other minor to major issues with the farm.

While the Health Analyzer can be useful, there are rules which are out of date or Health Analyzer warnings which cannot be resolved. As these rules are written into SharePoint’s codebase, it is not possible to modify the rules. We have the option of simply disabling them or ignoring them within Central Administration. An example of one of these rules is “Some content databases are growing too large.” This rule looks at the size of the database. If the database exceeds 100 GB, the health analyzer rule shows a warning. However, we know that Microsoft supports multi-terabyte content databases. The rule was created when mechanical hard drives were in common use. This warning was primarily designed for backup and restore scenarios, where it may not have been possible to back up or restore a database exceeding 100 GB in a reasonable amount of time. With the wide deployment of either SSD or flash-based systems, these databases may be restored in a matter of minutes rather than hours. It is still important to monitor database size, but this should be done outside of the context of SharePoint with SQL Server database monitoring tools.

If there are rules which are not required, they can be disabled via the Review rule definitions, as shown in Figure 19-14. Each rule will have an Enabled checkbox. Simply uncheck it to disable the rule. You may then delete the Health Alert from the Health Analyzer and the raised issue will no longer appear.

Performance Monitor for SharePoint

Performance Monitor may also be a useful tool for diagnosing server performance issues, such as examining outstanding ASP.NET requests, CPU usage by process, and so forth. The scenario in which Performance Monitor is used depends on the performance problem one is attempting to troubleshoot.

Performance Monitor for SQL Server

Performance monitoring of SQL Server can be quite in depth, but we will be skimming the surface here of “essential numbers.” For example, within the SQL Server Buffer Manager, Page Life Expectancy should be high. The value is measured in seconds; 300 seconds or higher is recommended in most systems. In addition, the Buffer Cache Hit Ratio should be well over 70 (or 70%). DMVs are also used to monitor SQL Server performance and are generally preferred over other methods.

Maintaining SharePoint database is also important. With SharePoint Server 2019, databases are set to auto-update statistics, but it is still good practice to implement a maintenance plan to manually update statistics on a periodic basis. In addition, a plan should be set in place to maintain database indexes. One popular script to handle these tasks is available from Ola Hallengren at https://ola.hallengren.com/ .