Previous Contents Index Next
Sun ONE Web Server 6.1 Administrator's Guide

> ## Chapter 10
> Using Log Files

> You can monitor your server’s activity using several different methods. This chapter discusses how to monitor your server by recording and viewing log files. For information on using the built-performance monitoring services, quality of service features, or SNMP, see Monitoring Servers.

> This chapter contains the following sections:

> - About Log Files
> - Logging on the UNIX and Windows Platform
> - Log Levels
> - About Virtual Servers and Logging
> - Redirecting Application and Server Log Output
> - Archiving Log Files
> - Setting Access Log Preferences
> - Setting Error Logging Options
> - Configuring the LOG Element
> - Viewing an Access Log File
> - Viewing the Error Log File
> - Running the Log Analyzer
> - Viewing Events (Windows)

> ##

> About Log Files

> Server log files record your server’s activity. You can use these logs to monitor your server and to help you when troubleshooting. The error log file, located in https-server_name/logs/errors in the server root directory, lists all the errors the server has encountered. The access log, located in https-server_name/logs/access in the server root directory, records information about requests to the server and the responses from the server. You can configure the information recorded in the Sun ONE Web Server access log file. You use the log analyzer to generate server statistics. You can back up server error and access log files by archiving them.

> Note | Due to limitations in the operating system, Sun ONE Web Server cannot work with log files larger than 2GB on Linux. As soon as the maximum file size is reached, logging will cease.

> ##

> Logging on the UNIX and Windows Platform

> This section discusses how log files are created. In addition, this section includes the following topics:

> - Default Error Logging
> - Logging Using syslog
> - Logging Using the Windows eventlog

> ### Default Error Logging

> On both the UNIX and Windows platforms, logs from the administration server are collected in the administration server https-admserve/logs/ directory. Logs from the server instances are collected in the https-server_name/logs/ directory.

> The default log level for the entire server can be set.You can redirect stdout and stderr to the server’s event log and direct the log output to the operating system’s system log. Additionally, you can direct stdout and stderr content to the server’s event log. Log messages by default are sent to stderr in addition to the specified server log file.

> Another feature available is to log the virtual server ID with the log message. This is a useful feature when multiple virtual servers are used to log messages to the same log file. You can choose to write the log messages to system log. When you do so, logging is not performed on the error log file. Instead the syslog logging service on UNIX, or the system logging service on Windows platform is used to produce and manage logs.

> You can also use the server.xml attributes to control the contents of this file. For details about the server.xml file, see the Sun ONE Web Server 6.1 Administrator’s Configuration File Reference.

> ### Logging Using syslog

> For stable operational environments where centralized logging is required syslog is appropriate. For environments where log output is frequently required for diagnostics and debugging, individual server instance or virtual server logs may be more manageable.

> Note | All logged data for the server instance and administration server in one file may prove difficult to read and debug. It is recommended that you use the syslog master log file only for deployed applications that are running smoothly. Logged message are intermixed with all other logs from the Solaris daemon applications.

> By using the syslog log file, in conjunction with syslogd, and the system log daemon, you can configure the syslog.conf file to:

> - Log messages to the appropriate system log
> - Write messages to the system console
> - Forward logged messages to a list of users, or forward logged messages to another syslogd on another host over the network

> Since logging to syslog means, logs from Sun ONE Web Server, and other daemon applications are collected in the same file, logged messages are enhanced with the following information to identify Sun ONE Web Server-specific messages from the particular server or virtual server instance:

> - Unique message ID
> - Timestamp
> - Instancename
> - Program name (webservd or webserv-wdog)
> - Process ID (PID of the webserv process)
> - Thread ID (optional)
> - Server ID

> The LOG element can be configured for both the administration server and the server instance in the server.xml file.

> For more information on the syslog logging mechanism used in the UNIX operating environment, use the following man commands at a terminal prompt:

> man syslog

> man syslogd

> man syslog.conf

> ### Logging Using the Windows eventlog

> For more information on the event log mechanism used in the Windows operating environment, refer to the Windows help system index for the keywords Event Logging.

> ##

> Log Levels

> The following table defines the log levels and messages in Sun ONE Web Server, in increasing order of severity.

> Table 10-1 Log Levels

> Log level | Description
> --- | ---
> finest finer fine | Messages indicate extent of verbosity of debug messages. finest gives the maximum verbosity.
> info | Messages are informative in nature, usually related to server configuration or server status. These messages do not indicate errors that need immediate action.
> warning | Messages indicate a warning. The message would probably be accompanied by an exception.
> failure | Messages indicate a failure of considerable importance that can prevent normal application execution.
> config | Messages relate to a variety of static configuration information, to assist in debugging problems that may be associated with particular configurations.
> security | Messages indicate a security issue.
> catastrophe | Messages indicate a fatal error.

> ##

> About Virtual Servers and Logging

> The Sun ONE Web Server can have virtual server instances. Each virtual server within a Sun ONE Web Server instance has its own identity and may have its own log file. The use of separate log files for each virtual server can help track server activity for particular transactions and resources.

> You can also direct logged messages from multiple virtual servers to one server log file. When so doing, you may wish to enable the logvsid in the LOG element of the server.xml file. This helps users to distinguish log messages originating from different virtual servers.

> <SERVER>

> ...

> <LOG file="/export//https-iws-files2.red.iplanet.com/logs/errors" loglevel="finest" logtoconsole="true" usesyslog="false" createconsole="false" logstderr="true" logstdout="true" logvsid="true"/>

> </SERVER>

> In this example, <LOG logvsid="true"> is responsible for including the virtual server ID in every log message. This allows you to differentiate messages coming from different virtual servers. The absence of attribute errorlog in the VS element, causes all the virtual servers to log messages to a single file.

> ##

> Redirecting Application and Server Log Output

> For developers, it is important that the application logs and server logs be made readily available during unit testing for Web application components and J2EE applications. On the Windows platform, developers prefer to see log messages displayed in a command window on the desktop. On the UNIX platform, many developers are comfortable with simply having the log messages stream to stderr in the terminal window in which the server instance is started, or, use the command tail -f to see the log messages written in log files.

> The server.xml file contains attributes that can be set for stdout and stderr to direct logged messages to a log file or to the terminal window, and so forth. See the Sun ONE Web Server 6.1 Administrator’s Configuration File Reference for more information on the use of stdout and stderr.

> ##

> Archiving Log Files

> You can set up your access and error log files to be automatically archived. At a certain time, or after a specified interval, your logs will be rotated. Sun ONE Web Server saves the old log files and stamps the saved file with a name that includes the date and time they were saved.

> For example, you can set up your access log files to rotate every hour, and Sun ONE Web Server saves and names the file “access.200307152400,” where name of the log file, year, month, day, and 24-hour time is concatenated together into a single character string. The exact format of the log archive file varies depending upon which type of log rotation you set up.

> Sun ONE Web Server offers the two types of log rotation for archiving files: Internal-daemon log rotation and Cron-based log rotation.

> ### Internal-daemon Log Rotation

> This type of log rotation happens within the HTTP daemon, and can only be configured at startup time. Internal daemon log rotation allows the server to rotate logs internally without requiring a server restart. Logs rotated using this method are saved in the following format:

> access.<YYYY><MM><DD><HHMM>

> error. <YYYY><MM><DD><HHMM>

> You can specify the time used as a basis to rotate log files and start a new log file. For example, if the rotation start time is 12:00 a.m., and the rotation interval is 1440 minutes (one day), a new log file will be created immediately when you save and apply changes regardless of the present time. The log file will rotate every day at 12:00 a.m., and the access log will be stamped at 12:00 a.m. and saved as access.200307152400. Likewise, if you set the interval at 240 minutes (4 hours), the 4 hour intervals begin at 12:00 a.m. such that the access log files will contain information gathered from 12:00 a.m. to 4:00 a.m., from 4:00 a.m. to 8:00 a.m., and so forth.

> If log rotation is enabled, log file rotation starts at server startup. The first log file to be rotated gathers information from the current time until the next rotation time. Using the previous example, if you set your start time at 12:00 a.m. and your rotation interval at 240 minutes, and the current time is 6:00 a.m., the first log file to be rotated will contain the information gathered from 6:00 a.m. to 8:00 a.m, and the next log file will contain information from 8:00 a.m. to 12:00 p.m. (noon), and so forth.

> ### Scheduler-based Log Rotation

> This type of log rotation is based on the time stored in the scheduler.conf file in the server_root/https-admserv/config/ directory. This method allows you to archive log files immediately or have the server archive log files at a specific time on specific days. The server’s scheduler configuration options are stored in schedulerd.conf in the server_root/https-admserv/config/ directory. Logs rotated using the scheduler-based method are saved in the following format:

> <original_filename>.<YYYY><MM><DD><HHMM>

> For example, access might become access.200307151630 when it is rotated at 4:30 p.m.

> Log rotation is initialized at server startup. If rotation is turned on, Sun ONE Web Server creates a time-stamped access log file and rotation starts at server startup.

> Once the rotation starts, Sun ONE Web Server creates a new time stamped log file when there is a request or error that needs to be logged to the access or error log file and it occurs after the prior-scheduled “next rotate time”.

> Note | You should archive the server logs before running the log analyzer.

> To archive log files and to specify whether to use the Internal daemon method or the scheduler-based method, use the Archive Log Files page in the Server Manager.

> ##

> Setting Access Log Preferences

> During installation, an access log file named access is created for the server. You can customize access logging for any resource by specifying whether to log accesses, what format to use for logging, and whether the server should spend time looking up the domain names of clients when they access a resource.

> To add %vsid% to the log file format string:

> 1. Access the Server Manager and choose the Logs tab.
> 2. Click the Access Log Preferences link.
> 3. Enter a new log file location and filename in the Log File: text box.
> 4. Click the Only Log: radio button.
> 5. Click the Virtual Server Id check box. Alternatively to this, you can click the Custom Format: radio button and add the string ‘%vsid%.

> Note | When adding the custom format string ‘%vsid%, you must use a new access log file.

> When changing the format of an existing log file, you should first delete/rename the existing log file OR use a different file name.

> Server access logs can be in Common Logfile Format, flexible log format, or your own customizable format. The Common Logfile Format is a commonly supported format that provides a fixed amount of information about the server. The flexible log format allows you to choose (from Sun ONE Web Server) what to log. A customizable format uses parameter blocks that you specify to control what gets logged. For a list of customizable format parameters, see the NSAPI Programmer’s Guide.

> Once an access log for a resource has been created, you cannot change its format unless you archive it or create a new access log file for the resource.

> You can specify logging preferences using the Access Log Preferences page in the Server Manager, or you can manually configure the following directives in the obj.conf file. In magnus.conf, the server calls the function flex-init to initialize the flexible logging system and the function flex-log to record request-specific data in a flexible log format. To log requests using the common log file format, the server calls init-clf to initialize the Common Log subsystem which is used in obj.conf, and common-log to record request-specific data in the common log format (used by most HTTP servers).

> For more information on the NSAPI logging functions, including valid directives and parameters, see the NSAPI Programmer’s Guide.

> ### Easy Cookie Logging

> Sun ONE Web Server has an easy way to log a specific cookie using the flexlog facility. Add “Req->headers.cookie.cookie_name” to the line that initializes the flex-log subsystem in the configuration file obj.conf. This logs the value of the cookie variable cookie_name if the cookie variable is present in the request’s headers, and logs “-” if it is not present.

> ##

> Setting Error Logging Options

> Sun ONE web Server 6.1 allows you to configure the information to be logged in the server’s errors logs.

> #### For the Administration Server instance

> 1. Access the Administration Server
> 2. Select the Preferences tab.
> 3. Click the Access Logging Options link.
> 4. Enter the required information.
> 5. Click OK and then Apply to save and apply the changes.

> #### For the Server Instance

> 1. Access the server instance.
> 2. Select the Logs tab.
> 3. Click the Error Log Preferences link.
> 4. Enter the required information.
> 5. Click OK and then Apply to save and apply the changes.

> ##

> Configuring the LOG Element

> The following table describes the attributes for the LOG element you can configure in the server.xml file:

> Table 2 LOG attributes

> Attribute | Default | Description
> --- | --- | ---
> file | errors | Specifies the file that stores messages from the default virtual server. Messages from other configured virtual servers also go here, unless the errorlog attribute is explicitly specified in the VS element.
> loglevel | info | Controls the default type of messages logged by other elements to the error log. Allowed values are as follows, from highest to lowest: finest, fine, fine, info, warning, failure, config, security, and catastrophe.
> logvsid | false | (optional) If true, virtual server IDs are displayed in the virtual server logs. These are useful if multiple VS elements share the same log file. Note that in Sun ONE Web Server 6.1 the logvsid element cannot be configured in the magnus.conf file.
> logstdout | true | (optional) If true, redirects stdout output to the errors log. Legal values are on, off, yes, no, 1, 0, true, false.
> logstderr | true | (optional) If true, redirects stderr output to the errors log. Legal values are on, off, yes, no, 1, 0, true, false.
> logtoconsole | true | (optional, UNIX only) If true, redirects log messages to the console.
> createconsole | false | (optional, Windows only) If true, creates a Windows console for stderr output. Legal values are on, off, yes, no, 1, 0, true, false.
> usesyslog | false | (optional) If true, uses the UNIX syslog service or Windows Event Logging to produce and manage logs. Legal values are on, off, yes, no, 1, 0, true, false.

> ##

> Viewing an Access Log File

> You can view the server’s active and archived access log files.

> To view the Administration Server’s access log from the Administration Server, choose the Preferences tab, and then choose the View Access Log page.

> To view an access log for the server instance from the Server Manager, choose the Logs tab, and then choose the View Access Log page

> To view an access log for an individual virtual server from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Access Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

> The following is an example of an access log in the Common Logfile Format (you specify the format in the Log Preferences window; see Setting Access Log Preferences for more information):

> wiley.a.com - - [16/Feb/2001:21:18:26 -0800] “GET / HTTP/1.0” 200 751 wiley.a.com - - [17/Feb/2001:1:04:38 -0800] “GET /docs/grafx/icon.gif HTTP/1.0” 204 342 wiley.a.com - - [20/Feb/2001:4:36:53 -0800] “GET /help HTTP/1.0” 401 571 arrow.a.com - john [29/Mar/2001:4:36:53 -0800] “GET /help HTTP/1.0” 401 571

> Table 10-3 describes the last line of this sample access log.

> Table 10-3 The fields in the last line of the sample access log file

> Access Log Field | Example
> --- | ---
> Hostname or IP address of client | arrow.a.com. (In this case, the hostname is shown because the web server’s setting for DNS lookups is enabled; if DNS lookups were disabled, the client’s IP address would appear.
> RFC 931 information | - (RFC 931 identity not implemented)
> Username | john (username entered by the client for authentication)
> Date/time of request | 29/Mar/1999:4:36:53 -0800
> Request | GET /help
> Protocol | HTTP/1.0
> Status code | 401
> Bytes transferred | 571

> The following is an example of an access log using the flexible logging format (you specify the format in the Log Preferences page; see Setting Access Log Preferences for more information):

> wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET /index.htm HTTP/1.0" "GET" "/?-" "HTTP/ 1.0" 304 0 - Mozilla/2.0 (WinNT; I) wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0 - Mozilla/2.0 (WinNT; I) wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0 - Mozilla/2.0 (X11; I; IRIX 5.3 IP22)

> ##

> Viewing the Error Log File

> The error log file contains errors the server has encountered since the log file was created; it also contains informational messages about the server, such as when the server was started. Unsuccessful user authentication is also recorded in the error log. Use the error log to find broken URL paths or missing files.

> To view the Administration Server’s error log file, from the Administration Server, choose the Preferences tab, and choose the View Error Log page.

> To view a server instance’s error log file, from the Server Manager, choose the Logs tab, and choose the View Error Log page.

> To view an error log for an individual virtual server, from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Error Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

> The following are two examples of entries in the error log; the first example shows an informational message indicating successful start up of the server, the second example indicates that the client wiley.a.com requested the file report.html, but the file wasn’t in the primary document directory on the server.

> [[22/Jan/2001:14:31:41] info (39700): successful server startup [22/Jan/2001:14:31:41] info (39700): SunONE-WebServer/6.1 BB1-01/22/2001 01:45 [22/Jan/2001:14:31:42] warning (13751): for host wiley.a.com trying to GET /report.html, send-file reports: can't find /usr1/irenem/ES60-0424/docs/report.html (File not found)

> ##

> Running the Log Analyzer

> The server-root/extras/log_anly directory contains the log analysis tool that runs through the Server Manager user interface. This log analyzer analyzes files in common log format only. The HTML document in the log_anly directory that explains the tool’s parameters. The server-root/extras/flex_anlg directory contains the command-line log analyzer for the flexible log file format. However, the Server Manager defaults to using the flexible log file reporting tool, regardless of whether you’ve selected common or flexible log file format.

> Use the log analyzer to generate statistics about your default server, such as a summary of activity, most commonly accessed URLs, times during the day when the server is accessed most frequently, and so on. You can run the log analyzer from Sun ONE Web Server or the command line. The log analyzer cannot generate statistics for virtual servers other than the default server. However, statistics can be viewed for each virtual server as described in Viewing an Access Log File.

> You must set the library path before attempting to run the flexanlg command line utility. The settings for various platforms are as follows:

> Solaris and Linux:

> LD_LIBRARY_PATH=server_root/bin/https/lib:$LD_LIBRARY_PATH

> AIX:

> LIBPATH=server_root/bin/https/lib:$LIBPATH

> HP-UX:

> SHLIB_PATH=server_root/bin/https/lib:$SHLIB_PATH

> Windows:

> path=server_root\bin\https\bin;%path%

> Note | Before running the log analyzer, you should archive the server logs. For more information about archiving server logs, see Archiving Log Files.

> To run the log analyzer from the Server Manager, follow these steps:

> 1. From the Server Manager, click the Logs tab.
> 2. Click Generate Report.
> 3. Fill in the fields.
> 4. Click OK.

> The report appears in a new window.

> For more information, see the Generate Report Page in the online help.

> To analyze access log files from the command line, run the tool, flexanlg, which is in the directory server-install/extras/flex_anlg.

> To run flexanlg, type the following command and options at the command prompt:

> flexanlg [ -P ] [-n name] [-x] [-r] [-p order] [-i file]* [ -m metafile ]* [ o file][ c opts] [-t opts] [-l opts]

> The following describes the syntax.

> flexanlg -h.): -P: proxy log format Default: no -n servername: The name of the server -x : Output in HTML Default: no -r : Resolve IP addresses to hostnames Default: no -p [c,t,l]: Output order (counts, time stats, lists) Default: ctl -i filename: Input log file(s) Default: none -o filename: Output log file Default: stdout -m filename: Meta file(s) Default: none -c [h,n,r,f,e,u,o,k,c,z]: Count these item(s) - Default: hnreuokc h: total hits n: 304 Not Modified status codes (Use Local Copy) r: 302 Found status codes (Redirects) f: 404 Not Found status codes (Document Not Found) e: 500 Server Error status codes (Misconfiguration) u: total unique URL's o: total unique hosts k: total kilobytes transferred c: total kilobytes saved by caches z: Do not count any items. -t [sx,mx,hx, xx,z]: Find general stats - Default:s5m5h24x10 s(number): Find top (number) seconds of log m(number): Find top (number) minutes of log h(number): Find top (number) hours of log u(number): Find top (number) users of log a(number): Find top (number) user agents of log r(number): Find top (number) referers of log x(number): Find top (number) for miscellaneous keywords z: Do not find any general stats. -l [cx,hx]: Make a list of - Default: c+3h5 c(x,+x): Most commonly accessed URLs (x: Only list x entries) (+x: Only list if accessed more than x times) h(x,+x): Hosts (or IP addresses) most often accessing your server (x: Only list x entries) (+x: Only list if accessed more than x times) z: Do not make any lists

> ##

> Viewing Events (Windows)

> In addition to logging errors to the server error log (see Viewing the Error Log File), Sun ONE Web Server logs severe system errors to the Event Viewer. The Event Viewer lets you monitor events on your system. Use the Event Viewer to see errors resulting from fundamental configuration problems, which can occur before the error log can be opened.

> To use the Event Viewer, perform the following steps:

> 1. From the Start menu, select Programs and then Administrative Tools. Choose Event Viewer in the Administrative Tools program group.
> 2. Choose Application from the Log menu.

> The Application log appears in the Event Viewer. Errors from Sun ONE Web Server has a source label of https-serverid or WebServer6.1.

> 3. Choose Find from the View menu to search for one of these labels in the log. Choose Refresh from the View menu to see updated log entries.

> For more information about the Event Viewer, consult your system documentation.

> Previous Contents Index Next

> Copyright 2004 Sun Microsystems, Inc. All rights reserved.