Generating Reports from Log Files
When a logging option is selected, you must decide how you would like to generate your log reports. Many options are available for generating reports with the different formats of log files. This section covers the various reporting technologies available, including some third-party utilities.
Site Server Express
Site Server Express can be used to analyze log files by using the Usage Import and Report Writer modules. The log files are imported into an Access database. The predefined reports provide traffic features, such as requests, visits, users, and organizations, that interact with the Web sites.
The first step in using Site Server Express is to import the log files. The following steps walk you through the configuration:
Start Usage Import.
Configure an Internet site using the Server Manager.
Create a data source with the appropriate log file type by right-clicking the Log Data Sources icon, and then click New.
Specify a server by right-clicking the scroll icon, and then click New.
Configure a site by right-clicking the newly created server icon, and then click New.
Open the Log file manager (from the File menu or from the toolbar), and type in the path to your log file or identify it using the Browse button.
Press the green Start import button on the toolbar to begin the import.
After the log files have been imported, use the Report Writer to generate a report. To accomplish this, do the following:
Start Report Writer.
Click the Create an analysis report button from the catalog option.
Select a report to run, such as Executive Summary.
Click Finish on the first option panel.
From the schematic representation of your report content, click the green Create report document button. The default is an HTML report, which will be opened with your browser upon completion.
Site Server Express is fairly simple to use. You can create custom reports, and you can generate reports not only in HTML format, but also in Microsoft Word or Excel formats. The task scheduler enables you to schedule the import and report generation, so after you have the servers and site configured, it's basically hands-off. If you don't want to purchase a third-party utility, Site Server Express is a good choice. I personally would give it a rating of 7 on a scale of 1 to 10 (with 1 being the low end and 10 being an excellent rating).
Usage Analysis
Usage Analysis is the parent of the Site Server Express log analysis tool. Usage Analysis is a component of Site Server and functions in much the same way as Site Server Express. The major differences are that you can use SQL Server as the database back end used to store the log files, and WHAT contains 45 predefined reports.
Our company was contacted by the Site Server team at Microsoft to do some stress testing of this product at one point. Microsoft provided their own hardware and software and helped get things set up initially. Just so that you have an idea of hardware recommendations, the server that Microsoft provided was a quad-processor with 512MB of RAM, and the SQL database device was set at 17GB—and this was just for log file analysis. We had configured Usage Analysis to import log files from and run reports for about 50 Web sites.
Even with the power of this server, we still found that performance was a concern. Importing the log files was not a problem, but generating the reports could take hours. One site generated log files that averaged 60–70MB per day. The import would take about one hour, but generating a report for two weeks' worth of files took more than 26 hours. However, when the process was finished, the reports were exactly what we wanted. Weighing the performance issues with the great reports, I give Usage Analysis a rating of 6 out of 10 points.
Access Watch
Access Watch is an analysis and report-generating utility that uses Perl. If you already have Perl running on your server, this is a great little program to generate some basic statistical information. It is available for download from www.accesswatch.com. After you configure the product and try it out, you can purchase a license for well less than $100.
When our company purchased Access Watch, we had it running within an hour of downloading the source files. This utility is incredibly easy to configure and gives you the basic statistics for Web site traffic. The reports are HTML pages that can be accessed from anywhere on the Internet if you have a valid account configured for accessing them. The pages include a summary of activity on the site for the specified period, the accesses for specific pages, hourly traffic, daily traffic, and a breakdown of domains and hosts. We created simple batch files and scheduled these to run with the Task Scheduler included with both Windows NT and Windows 2000. For the price and ease of use, I give Access Watch 8 out of 10 points.
WebTrends
WebTrends is an easy-to-use Windows program. It is also easy to configure and can be scheduled to import the logs and run reports. The graphical reports can be customized. WebTrends is not as fast as Access Watch when generating reports, but it is a lot faster than Usage Analysis. The interface is very basic, and setting up Web sites for reporting is a snap. Plug-ins also are available with WebTrends for use on Internet Information Services servers. These plug-ins can be used to do reverse DNS lookups and to track cookies, which is very useful if you have Web sites that are using cookies to track users and if you prefer to see readable names rather than IP addresses.
WebTrends was our software of choice both before and after the Usage Analysis trial. However, when I upgraded to version 4.2 and started running reports, I noticed a couple blank fields. This caused errors in the reports for the number of hits to a Web site's home page. The technical support department was unfortunately not very helpful in trying to resolve the issue, forcing us to shop around for another solution. In the mean time, WebTrends released version 4.5, and the new version has resolved the issue. WebTrends can be found at http://www.webtrends.com/. I give it 7 out of 10 points.
Statistics Server
Statistics Server from MediaHouse Software Inc. analyzes Web site traffic in real time, so it generates Live Stats reports in a nice, simple Web interface. It includes many specialty reports and can store up to one full year of historical data that can be queried any time and in any format. In contrast to other statistics software that runs reports at a scheduled time, Statistics Server analyzes log files in near real time. The greatest benefit to this for our company is that our servers are no longer burdened for a time period once a day while the software runs the reports. This software distributes the load evenly over 24 hours. With only a few clicks of the mouse, a user can request a daily, monthly, or even yearly report for a Web site.
Statistics Server was created for marketing people and Web administrators who analyze site traffic to make sure their Internet and intranet investments work. I like it because this utility works on a single Web site or a server that hosts a few hundred Web sites. We use Statistics Server to respond to all our clients' reporting requests, instead of using different statistics software for different reports.
I recommend installing Statistics Server software on a separate computer than the Web server because it is constantly processing, and you don't want it to affect the performance of the Web sites it is analyzing. Statistics Server reads log files for different Web sites across a TCP/IP network on short, timed intervals—in our case, every six minutes. It processes Web site log files in real time and stores the resulting data in a statistics database that is much smaller than the original log files. Web site owners can then query the database from the Statistics Server HTML interface.
A few of the reasons that we have chosen Statistics Server include these:
Web-based interface is easy to use and is accessible from nearly every Web client.
Multiple people can concurrently access statistics for one or more Web sites.
The utility can generate reports based on daily, weekly, or monthly time periods, or any specific length of time.
The utility provides detailed graphs and tables.
The Who's On feature shows who is on the Web site at that moment and what those users are accessing.
The utility tracks inbound and outbound advertise click-throughs.
The utility can generate a printable HTML page.
Statistics Server comes with licenses for 500 Web sites and can be used on one or more servers. We now can use the same software for all our clients, and they determine which set of statistics that they need without calling us to generate a new report and schedule it for them. You can review or purchase Statistics Server at http://www.mediahouse.com/. This utility deserves 9 out of 10 points.