#LyX 1.1 created this file. For more info see http://www.lyx.org/
\lyxformat 218
\textclass article
\language english
\inputencoding latin1
\fontscheme palatino
\graphics default
\paperfontsize default
\spacing single 
\papersize Default
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\paperorientation portrait
\secnumdepth 3
\tocdepth 3
\paragraph_separation skip
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default

\layout Title

Ipfcontrol db-backend Design
\layout Author

Tycho Fruru (tycho@fruru.com)
\newline 
Conostix SA (tycho.fruru@conostix.com)
\layout Date


\size footnotesize 
$Id: db-backend-design.lyx,v 1.3 2001/12/17 15:21:07 fpmip Exp $
\layout Part

db-backend in the ipfc infrastructure
\layout Standard

Ipfc is a complete distributed management, logging and monitoring infrastructure
 for (mostly) security components.
 The architecture consists of the following elements :
\layout Enumerate

dr-server : this is the central data repository, based on an Apache server
 with some external modules.
 All ipfc data (except urgent alerts) will pass through this server.
 So, configurations, logs and monitoring info is normally dispatched using
 the dr-server.
\layout Enumerate

db-backend : this is a searcheable repository for logging and monitoring
 information.
 The db-backend will get its information from the dr-server and put the
 (parsed) info in a database.
\layout Enumerate

agent : this is the wrapper which is put around a remote software package
 (such as a webserver or firewall) and which interfaces it with the dr-server
 for getting configurations, sending logs and updating monitoring information.
\layout Enumerate

reporter : generates 
\begin_inset Quotes eld
\end_inset 

nicer
\begin_inset Quotes erd
\end_inset 

 reports of the raw event data stored on the db-backend.
\layout Enumerate

frontend : offers a web-based user interface of the information stored in
 the db-backend.
 
\layout Part

db-backend design
\layout Section

Introduction
\layout Standard

The current db-backend is built around a postgresql database.
\layout Standard

The database must be able to cope with a multitude of logging information
 (some of which will be binary, some free-text, some formatted text).
 Moreover, this logging information will be vastly different between the
 different information sources - a firewall generates different logging
 information than a webserver or an IDS.
 This makes defining a 
\begin_inset Quotes eld
\end_inset 

complete
\begin_inset Quotes erd
\end_inset 

 logging infrastructure very difficult.
 
\layout Standard

The approach IPFC takes is to have one table 
\begin_inset Quotes eld
\end_inset 

events
\begin_inset Quotes erd
\end_inset 

 which contains data which is common to all types of events.
 This table can then be specialized in several other tables which contain
 more information.
 The current implementation using postgres enables this quite elegantly
 using the 
\begin_inset Quotes eld
\end_inset 

inherits
\begin_inset Quotes erd
\end_inset 

 option whilst creating the tables, but in other databases it is also possible
 using views.
\layout Section

Database design
\layout Standard

The database is meant to be flexible but is basically built around a fairly
 small number of tables :
\layout Enumerate


\emph on 
agents
\emph default 
, which describes the various installed agents (type of agent, location,
 ...)
\layout Enumerate


\emph on 
logfiles
\emph default 
, which contains a reference to each and every XML file processed by the
 db-backend (for retracing and reliability monitoring)
\layout Enumerate


\emph on 
events
\emph default 
, which describes the content of the XML files that have been processed
 by db-backend
\layout Standard

These are the essential tables, but there are a number of auxiliary tables
 which are meant to support these basic tables.
\layout Enumerate


\emph on 
agent_types
\emph default 
, which describes the various different types of agent (eg.
 apache, syslog, snort, ...)
\layout Enumerate


\emph on 
agent_groups
\emph default 
, to put multiple agents in multiple groups (n:n)
\layout Enumerate


\emph on 
agent_group_descriptions
\emph default 
, to describe the different groups defined numerically in 
\emph on 
agent_groups
\emph default 
.
\layout Enumerate


\emph on 
agent_url_ids
\emph default 
, to link agents to the location on the dr-server where their events are
 pushed.
 This relation is used to find the right section in the 
\emph on 
db-wrapnet.pl
\emph default 
 configuration script.
\layout Standard

The system as described above cannot cope with the (very) different log
 file formats, alerts, etc that we wish to process.
 The 
\emph on 
events
\emph default 
 table at most allows you to add a free-form text message to every event.
 It is impractical and inefficient to try to extract data on a higher level
 from the database using only the information available in 
\emph on 
events.
 
\emph default 
We would eg.
 like to select all records (from different agents) which have the same
 IP source address, irrespective of whether the event was generated by a
 NIDS, a firewall, or a router.
\layout Standard

To make this possible, we inherit the basic columns of 
\emph on 
events
\emph default 
, adding more and more features.
 The currently implemented 
\begin_inset Quotes eld
\end_inset 

supersets
\begin_inset Quotes erd
\end_inset 

 of 
\emph on 
events
\emph default 
 are:
\layout Enumerate


\emph on 
events_with_src_addr
\emph default 
, containing a source host address or name
\begin_deeper 
\layout Enumerate


\emph on 
events_with_src_addr_port
\emph default 
, containing also a numeric IP protocol number and a port number
\layout Enumerate


\emph on 
events_with_src_dst_addr
\emph default 
, containing also destination addresses
\begin_deeper 
\layout Enumerate


\emph on 
events_with_src_dst_addr_port
\emph default 
, containing also source and destination port numbers, and a numeric IP
 protocol number
\begin_deeper 
\layout Enumerate


\emph on 
events_firewall
\emph default 
, containing information pertinent to firewalls (eg.
 was the packet accepted or rejected)
\end_deeper 
\end_deeper 
\end_deeper 
\layout Enumerate


\emph on 
apache_access_log_1
\emph default 
, a format for apache access logs
\layout Enumerate


\emph on 
syslog_events_1
\emph default 
, a format for syslogs
\layout Enumerate


\emph on 
availability_1
\emph default 
, for simple availability messages (no specific data)
\the_end
