[NBLUG/talk] Cobalt server and parsing access logs

[Ron Wickersham]

On Sat, 22 May 2004, Todd Cary wrote:

---snip---

> however what is the type of log file do I have?
>
>
> 67.69.7.202 - - [21/May/2004:05:18:59 -0700] "GET / HTTP/1.0" 200 18276
> "http://www.google.ca/search?q=yacht+club+&hl=en&lr=&ie=UTF-8&start=10&sa=N"
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

---snip---

this type of log is called the "combined" format:

67.69.7.202
   the ip address (or host name if dns processing is turned on),

-
   the ident from that host (if ident processing is turned on -- but it never
      is these days on the public internet,

-
   the "authorized user" so you'll never see anything here unless you have
      password protected portions of your web server,

[21/May/2004:05:18:59 -0700]
   the time the request was processed,

"GET / HTTP/1.0"
   the actual request sent by http coming from the client (since the
      field separator is a space, double quotes are used for fields that
      have embedded spaces,

200
      status code returned to the client (200 means ok),

18276
      the number of bytes sent to the client,

###  if the log entry line ended here, the format would be "common" transfer
     log and would be stored in the file conventionally called access_log,

"http://www.google.ca/search?q=yacht+club+&hl=en&lr=&ie=UTF-8&start=10&sa=N"
     this is the "referral" entry which would contain the link that was
     click on by the client to reach the requested page.   if this were
     not combined format, this entry could optionally be found in a file
     called referer_log,

"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
     and the final entry is the browser response sent by the client, and
     if this were not combined format, then this entry could optionally be
     found in a file called agent_log.

note that although your log entries were separated into more than one line
by your mail user agent, the actual log contains a single line for for the
above information, each field separated by a space (thus empty fields have
to be represent by something so they chose a hyphen and fields with spaces
inside are enclosed by double quotes.

in addition to "common" and "combined" apache makes provision for "custom"
logs where you can freely choose what and in which order to log, but its
rare these days to see anyone use anything but "combined"  (which is
actually a custom definition in current Apache configuration files
(custom and common are not hard coded in current Apache releases but were
hard coded in the httpd server written at the httpd project at National
Center for Supercomputer Applications (NCSA) at University of Illinois,
Urbana Champaign.  the httpd server is the code that was patched by the
Apache group.

-ron

--
/~\  The ASCII Ribbon Campaign
\ /    No HTML/RTF in email
 X     No Word docs in email
/ \  Respect for open standards