What's new in version 1.0

# What's new in version 1.0?

OOBA Summary

  A new report section has been added which will display information regarding out of band authentication. See the configuration file option ooba_summary.

Workaround for Raptor SMTP received bytes bug

  Reptor can work around a bug in Raptor that occasionally produces impossibly huge values for the number of bytes received in some logfile entries for the SMTP protocol. This workaround is disabled by default and can be enabled with the new configuration file option smtp_bugfix.

DNS caching

  Reptor now maintains a local cache of DNS names, in order to avoid redundant (and probably slow) lookups. This will dramatically speed up processing if you're using "dns all", which you shouldn't be doing. If you're using "dns print", the impact is not as significant.

New HTML themes

  Four new HTML themes have been added: "Aqua" (based on the MacOS X GUI), "Hotmail2" (based on the new Hotmail design), "Slashdot" (based on slashdot.org), and my new favorite, "Slashcode" (based on slashcode.com). Themes may not render correctly if your browser is deprecated and does not properly support modern HTML standards. All themes produce valid HTML 4.01 Transitional output and have been verified to work properly with Internet Explorer 5.0, 5.5, and 6.0, Mozilla 0.9.4, and Opera 5.12. I've had enough of trying to get the themes to render properly with Netscape 4.x. If you insist on using it, don't email me to complain about how crappy your output looks.

Bugfixes

  Fixed the "dns print" bug that caused processing to run for extremely long times when that option was specified and DNS resolution was not enabled on the firewall. (It was actually functioning the same way that "dns all" does.)

  Fixed a bug that caused the alert section user column to show the string "$user" instead of the actual user name.

# What's new in version 0.99?

Automatic handling of split logfiles

  Woohoo! Finally, Reptor can automatically handle large logfiles that have been split by the firewall and named logfile.date, logfile.date-1, logfile.date-2, etc. This new feature can work with files retrieved by remotelog and even in conjunction with the uncompress configuration file option, in the case that each logfile chunk is individually compressed.

basedir option

  The --basedir command line option or the basedir configuration file option may be specified to indicate the Reptor install directory. This may be helpful when running Reptor from a command scheduler, so that it can properly locate the required files. If both the command line and configuration file options are specified, the command line option takes precedence. If --basedir is specified and your configuration file is named reptor.cfg, you won't need to specify the --config command line option.

HTML themes

  Reptor now uses a simple theme system to format the output. This makes it very easy to change the look of the reports if so desired, without the need for editing the main program itself. A number of sample themes are included with the distribution. Selecting a theme is done with the new configuration file option theme. As a result of this new feature, the following configuration file options have been removed: table, highlight, and style_sheet. All included themes produce "HTML 4.01 Transitional" compliant output, which has been tested to render properly with Internet Explorer 5.5, Netscape Navigator 4.76, Opera 5.02, and Mozilla 0.8.

  Note: If you make your own theme and send me a copy, I will include it in the base distribution of the next release. Please make sure your theme produces "HTML 4.01 Transitional" compliant output.

User WWW summary

  A new report has been added that shows WWW usage by user. For each user, the most popular web sites they have visited will be summarized. This option is only useful if users authenticate with the firewall. The new configuration file option user_www_summary controls this report.

--history command line option

  If you've specified the history_file option in your configuration file, Reptor will normally only update history when processing the previous day's logfile, to avoid contaminating the history file in the case that you process a logfile more than once. Specifying the --history command line option will override this behavior, and Reptor will update the history regardless of the logfile being processed. It is up to you to keep the history file in date order.

Support for Raptor Mobile v6.5

  Reptor will now detect activity from Raptor Mobile version 6.5 clients in the mobile_summary section.

WebNOT fetcher summary

  A new section has been added that shows the status of the WebNOT fetcher process. It will report if the ratings file is up to date or if a new one has been been installed. This report section is controlled by the new configuration file option fetcher_summary.

Minor text formatting updates

  Reptor can now print large numbers comma formatted. This optional formatting is controlled by the configuration file option nice_format, which replaces the nice_bytes option.

  Reptor will now print "Gb" instead of using huge "Mb" values.

Minor bug fixes

  The uncompress configuration file option now works in conjunction with the --log command line option.

  Wide tables are now formatted correctly by Netscape. (This wasn't a bug in Reptor, but work-around for the issue has been discovered.)

# What's new in version 0.98?

Protocol detail by host

  A new report section has been added that shows traffic for a single protocol, grouped by host. Multiple sections can be added, each reporting on a different protocol. Such a section could answer the question, "What hosts generate (or serve) the most ftp traffic?" or "What hosts generate (or serve) the most smtp traffic?"

Protocol detail by user

  A new report section has been added that shows traffic for a single protocol, grouped by user. Multiple sections can be added, each reporting on a different protocol. Such a section could answer the question, "What users generate (or serve) the most ftp traffic?" or "What users generate (or serve) the most smtp traffic?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

User detail by host

  A new report section has been added that shows traffic for a single user, grouped by host. Multiple sections can be added, each reporting on a different user. Such a section could answer the question, "What hosts is Joe visiting (or serving to)?" or "What hosts is Sam visiting (or serving to)?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

User detail by protocol

  A new report section has been added that shows traffic for a single user, grouped by protocol. Multiple sections can be added, each reporting on a different user. Such a section could answer the question, "What protocols is Joe using?" or "What protocols is Sam using?". Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

Host detail by protocol

  A new report section has been added that shows traffic for a single host, grouped by protocol. Multiple sections can be added, each reporting on a different host. Such a section could answer the question, "What protocols is 192.168.1.1 using?" or "What protocols is 192.168.1.2 using?"

Host detail by user

  A new report section has been added that shows traffic for a single host, grouped by user. Multiple sections can be added, each reporting on a different host. Such a section could answer the question, "What users are accessing 192.168.1.1?" or "What users are accessing 192.168.1.2?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

Bug fixes

  Fixed a Unix bug where a single digit day is reported without zero padding.

  Fixed a possible divide by zero error in the history summary.

  Fixed a bug where the message summary would default to no messages instead of all messages when a severity limit was not specified.

Minor usability enhancements

  Removed the Netscape-choking font size definitions from reptor.css.

  Added better checking for invalid configuration file options.

  Fixed some HTML formatting inconsistencies.

# What's new in version 0.97?

Message summary

  Reptor can now print a summary of messages other than statistical information (type 121). This can be useful for determining, at a glance, possible abuse of the firewall. For example, possible port scans (type 347), unauthorized protocol commands (type 334), or attempts to access control ports (type 515) might warrant further investigation.

Historical activity summary

  Reptor can now accumulate daily activity statistics in a history file. A summary has been added that graphs the contents of this file. Alternatively, it could be easily graphed with external programs such as gnuplot or Excel.

Color highlighting

  Reptor can now highlight alternating table lines with different colors. This is done with style sheet classes (called odd and even) for the TD attribute.

Minor usability enhancements

  Reptor no longer requires an explicit "+" on ascending sort identifiers in the configuration file.

  Reptor will exit with an error if it encounters an unprocessable configuration file directive.

Bug fixes

  Fixed a spelling error.

  Fixed the mail output subject line when processing cut logfiles.

  Fixed a bug where the user summary wouldn't work unless the alert section was also included.

# What's new in version 0.96?

Logfile compression

  Reptor can now read compressed logfiles. The decompression program can be specified with the uncompress configuration file option. Also, files that are saved (as specified by the save_logfile option) can be compressed. The program to perform this compression can be specified with the compress option.

Support for cut/merged logfiles

  The firewall creates each new logfile with a specifically formatted first line. Previously, Reptor required this line to be present. This caused problems if logfiles were cut into multiple pieces or if multiple logfiles were merged into a single large one. Reptor now specifically supports these situations by not requiring this special first line, and gracefully adapting the output if it is not present. A new command line option --ignore can be specified to indicate that the logfile to be processed has been cut or merged. This will cause Reptor to not look for the first line, and ignore it if present. This feature allows Reptor to report on a time period less than one day by feeding it a cut logfile, or on a time period greater than one day by feeding it merged logfiles.

  Normally, if output file or output ftp is specified without a filename, Reptor will use the datestamp from the first line of the logfile to automatically generate one. If this situation exists when the --ignore option is specified, Reptor will default to a filename of "reptor.html".

Verify save_logfile directory

  If the save_logfile option is specified, Reptor will verify that the specified directory actually exists.

# What's new in version 0.95?

GNU GPL

  Reptor is now released under the GNU General Public License.

User summary

  A new summary has been added that shows usage by user. This is typically only useful if users authenticate against the firewall.

Top level domain summary

  A new summary has been added that shows usage by top level domain.

Raptor Mobile summary

  A new summary has been added that shows the number of connections made by Raptor Mobile clients. Since there is no way to determine when the tunnel was closed, it is not feasible to report on the tunnel traffic itself -- only the number of connections made is shown.

Save local copy of logfile

  If you're using remotelog to retrieve logfiles, you can specify the new save_logfile option in the configuration file to indicate that you want Reptor to make a local copy of the logfile for additional analysis or backup.

New path parameter for remotelog option

  If you are using the remotelog utility to retrieve logfiles from your firewall, and the remotelogfile executable is not in your shell's PATH, you'll need to specify this so Reptor knows where to find it.

Check for remotelogfile program

  If the remotelog option is specified in the configuration file, and Reptor is run with the --verify command line parameter, Reptor will abort with an error if the remotelogfile program does not exist or is not executable.

Passive FTP transfers

  The FTP output now uses PASV mode for compatibility with Raptor v6.

New options for email output

  New mail_from and mail_server options in the configuration file allow you to specify the from address and SMTP relay if you're using email output. This may be necessary to avoid problems with anti-spam features of some email servers.

Style sheet option

  A new style_sheet option in the configuration file allows you to specify a style sheet to be embedded in the report.

Bug fix

  Fixed code to recognize directory names ending with \ and not just /.

Version option

  A new --version command line option has been added. Guess what it does.

# What's new in version 0.91?

Support for Raptor Firewall version 6

  Reptor now understands the format of logfiles generated by Raptor Firewall version 6.

WebNOT ratings alert

  Reptor can be configured to generate an alert message for connections that trigger a WebNOT ratings denial. This can be specified with the new configuration file option ratings.

New fields available in alert section

  The alert section can now optionally contain the logfile fields "rule", "id", "op", and "result". These are specified with new configuration file options show_rule, show_id, show_op, and show_result.

# What's new in version 0.90?

Extended configuration file options

  Almost all command line options have been replaced with configuration file options. No more huge command lines! A few command line options remain in order to allow overrides of the configuration file settings. These remaining options have been changed to the GNU style long format. For example, -d is now --date. Refer to the About Reptor page for details.

Automatic local/remote host discovery

  The configuration file local option has been removed in favor of the interface option. Instead of providing a list of IP addresses that are considered to be "local", you must specify what network interfaces exist in the firewall server. Reptor will then automatically determine which hosts are local and which are remote based on what interface they reside behind.

Traffic filter by interface and direction

  Traffic can now be filtered by direction and by interface. If you have more than two network interfaces in the firewall server, you may want Reptor to ignore certain traffic, such as "from the DMZ interface to the outside interface" or "from the inside interface to the DMZ interface".

Remotelog support

  Reptor can now obtain logfiles through the remotelog utility that is provided with the firewall. This allows Reptor to be run on a machine other than the firewall server without the need for customized scripting to transfer the logfiles, and without the fear of the logfile content being sniffed from the network.

Total summary

  A new simple summary has been added that shows total traffic through the firewall.

Alert summary

  A new simple summary has been added that shows how many alerts each host has triggered. This makes misuse tracking a bit easier -- a host that triggers two or three alerts is probably nothing to worry about, but a host that triggers fifty seven calls for further scrutiny.

Time of day summary

  A new summary has been added that shows the amount of traffic based on the time of day.

Time of day alert trigger

  Logfile entries can now trigger alerts based on the time of day, and can be separately defined for each protocol.

Graphs

  Summary tables can now optionally include bar graphs to indicate traffic as a percent of total.

Configurable word search fields

  You can now specify which fields are included in the word search. The string that the word search scans can be built from any combination of source host name, remote host name, and connection argument. (The connection argument is the value from the logfile "arg" field, which typically contains the URL.)

Reverse DNS lookups minimized

  Reverse DNS lookups can now optionally take place only if a host name actually gets included in the output. Doing this is dramatically faster than performing a lookup on every unresolved IP address, and only slightly slower than not doing any reverse lookups at all. However, the drawbacks of this option are that the hostname will not be able to be included in the word search string, and that summaries will not be able to be sorted by hostname.

Default style sheet

  The HTML output now includes settings defined by a default style sheet, to allow easy customization of report aesthetics.

Optional hyperlinks

  Hyperlinks are now optional. New links have also been added before each report section to allow easy navigation.

Automatic version detection

  Reptor will now automatically determine the firewall version you are using.

Time alert renamed

  The time alert option has been renamed to duration.

Size alert renamed

  The size alert option has been renamed to volume.

Bug fixes

  Summaries now correctly count the limit of number of entries, and durations greater than 23:59:59 will now display correctly.

Additional optional alert section fields

  The alert section may now contain additional optional fields, individually selectable. These include: the reason for the alert, the user, the argument of the connection, and the authentication method used.

Removed non-HTML output formats

  The plain text and delimited text output format options have been removed.

Allow aliases with spaces

  Aliases may now contain spaces.