Separate by User (SBU)

sbu-1.0
By Steve Coile <scoile@patriot.net>

Contents

Summary

This Perl program will separate combined log files into separate log files on a per-user basis (i.e. into a log file for each user).  The utility recognizes a variety of log file formats commonly found on RHL systems.

Requirements

In order to make use of this program, you will need to have Perl 4 or higher installed.  Red Hat Linux 4 systems should be using the perl-5.003-8 package provided by Red Hat Software (available as an update for 4.1 and 4.0).

Installation

In order to use this program, download the following file and place it in the /usr/local/bin directory:

http://www.patriot.net/users/scoile/isp-redhat/log-processing/sbu-1.0

If you have Lynx installed, the following commands will download the file and place it in the proper location:

SOURCE="http://patriot.net/~scoile/isp-redhat/log-processing/sbu-1.0"
TARGET="/usr/local/bin/sbu"
lynx -source "$SOURCE" > "$TARGET"

After placing the file in /usr/local/bin, change the permissions to allow execution:

chmd a+rx /usr/local/bin/sbu

Configuration

No configuration is necessary.

Operation

sbu accepts a log file via standard input, locates any username reference in each line, and outputs each line to a separate log file specific to the user named.  Usernames are recognized by their positions within the log entry.  The sbu utility recognizes several log file formats in common use and knows where to find usernames within them.  Log file formats may be selected with the "-f" switch.

User-specific log files are created in the current working directory and are named "user.log", where "user" is replaced with the user's username.  If a file already exists by that name, the log entries at appended to the existing file.  The user-specific log file naming convention may be changed with the "-l" switch.

Log entries that do not refer to a particular user are printed to standard output and otherwise ignored.

Additional information on command line options is available by invoking sbu with the "-h" switch.

History

PatriotNet offers it's user the ability to publish their Web pages on our Web servers.  However, we limit the number of hits our users' Web pages may get per month.  If a user's Web pages receive more hits than the monthly limit, we charge the user an additional fee.

This policy requires user-by-user Web service accounting.  In order to allow quick access to individual user statistics, we decided that the best way to obtain and access this accounting information was to separate the server-generated log file into user-specific log files.  At the end of the month, we can delete the log files for users below the limit and keep the ones for users over the limit.  Keeping the log files is important in the event that a user disputes the extra fee.

sbu started as split-by-user.pl, which only recognized the common log format used by the Apache HTTPd Web server.  I quickly realized, however, that per-user information from a variety of log files could be worthwhile, so I generalized the program and expanded the command line interface slightly, yeilding sbu v1.0.