In these days of shrinking budgets there doesn’t seem to be any less emphasis on providing timely user experience information, or at least monitoring web site traffic. I’m surprised to find how many users don’t avail themselves of an outstanding source of free information, monitoring IIS Extended Logs. It amazes me to see how much time, effort, and money are spent engineering methods to sample or capture the time it takes to get pages, which pages are gotten the most, how many Bytes are being transferred, and which pages referred users to the big hitting pages. All this information is available in the IIS Extended Logs.

IIS Log Monitoring is a snap to set up. First, not all the useful statistics named above are captured by default. Selecting what you want to be logged is easily configured in the Extended Logging Properties. The controls are under the Web Site properties on Win 2003 Server and under Site\Logging on Win 2008 Server. You can select the fields that you want to keep track of. So all you have to do is click on the check box beside Time Taken and Sent Bytes to start logging that data. The best part is that you don’t even have to stop or restart IIS. IIS starts using the new logging format as soon as you save the changes.

Why you might care about some info:

• Server data - This is useful for understanding load in a clustered or load-balanced environment, e.g. which server or IP served which pages,
• HTTP Status (scStatus) - Server Client Status is the numeric status of the operation, for example, a Get of a page may yield a status of 200 (OK) or 404 (Page Not Found),
• Bytes Sent and Received - from the server perspective; this is best converted to a larger number such as KBytes or even MBytes for IO intensive sites,
• Time Taken - The length of time the action (e.g. a GET) took in milliseconds; I usually convert this to seconds, and set a threshold to capture slow pages,
• Referrer - The site the user last visited or provided a link to the current page; this is not always populated depending on how the user got the page,
• Port - only needed if you serve both HTTP and HTTPS, and want to differentiate between the two connection types.

Why you might not care about other info:

• User Name - this is normally not captured for HTTP connections, but could be useful for HTTPS connections,
• URI Query - only used for dynamic pages, and often not used, but is available,
• Win32 Status - non-intuitive data that doesn’t add to the Server’s HTTP status,
• Protocol Status - sub-status error code; I’ve never used it,
• User Agent - this is the browser the user employed to view your site, and is only useful when debugging compatibility issues,
• Cookie - this is a very specialized need rarely used, and people always know when they need to use it,
• csHost - Client host - will not usually be captured, and is largely irrelevant to aggregated views of web service.

You can parse the IIS logs with a FOR Loop in a Windows BAT file that counts the unique page hits, plus accumulates the Time Taken data to calculate Max, Min, and Avg times for each page. Or, you can send me an email, and I’ll give the Longitude solution I recently created for IIS web traffic monitoring. Longitude has a built in File Parser that I used for the collector. Even better, I configured the collector to set a pointer so that it only reads new entries in logs that can be really huge. I also created a few reports to go with the collector: Page Hits Summary, Slow Pages, and Page Referral Analysis, which I’ll be happy to pass along…