#LyX 1.1 created this file. For more info see http://www.lyx.org/
\lyxformat 218
\textclass article
\begin_preamble
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead[LE,RO]{\textit{IPFC protocol and data definition}}
\fancyfoot[LE,RO]{}
%\pagenumbering{roman}
%\title{\vspace*{100pt}\Business Plan - Conostix\normalsize}
%
% You might want to change this :
%
\author{Conostix S.A.}
\end_preamble
\language english
\inputencoding auto
\fontscheme pslatex
\graphics default
\paperfontsize 10
\spacing single 
\papersize a4paper
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\paperorientation portrait
\secnumdepth 5
\tocdepth 3
\paragraph_separation skip
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default
\bullet 1
	0
	9
	4
\end_bullet
\bullet 2
	0
	10
	-1
\end_bullet

\layout Title

IPFC v1.0.1-DRAFT Protocol Specification and Data-format Specification 
\layout LaTeX


\backslash 
pagebreak
\layout LyX-Code

Copyright (c) 2000,2001 Alexandre Dulaunoy, adulau@thinkingsecure.com
\layout LyX-Code

Copyright (c) 2001 Tycho Fruru, tycho@fruru.com
\layout LyX-Code

Copyright (c) 2001 Conostix S.A.
 Luxembourg
\layout LyX-Code

 
\layout LyX-Code

This program is free software; you can redistribute it and/or modify it
 
\layout LyX-Code

under the terms of the GNU General Public License as published by the 
\layout LyX-Code

Free Software Foundation; either version 2 of the License, or (at your 
 
\layout LyX-Code

option) any later version.
 See <http://www.fsf.org/copyleft/gpl.txt>.
 
\layout LyX-Code

This program is distributed in the hope that it will be useful, but 
\layout LyX-Code

WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
 
\layout LyX-Code

or FITNESS FOR A PARTICULAR PURPOSE.
 See the GNU General Public License 
\layout LyX-Code

for more details.
\layout LyX-Code

 
\layout LyX-Code

\layout LyX-Code

Conostix S.A.
\layout LyX-Code

Technoport Schlassgoart
\layout LyX-Code

66, rue de Luxembourg
\layout LyX-Code

L4221 Esch-Sur-Alzette
\layout LyX-Code

Grand-Duchy of Luxembourg
\layout LyX-Code

mailto: support@conostix.com url: http://www.conostix.com/
\layout LyX-Code

 
\layout LyX-Code

 
\layout LyX-Code

$Version$
\layout Date

\SpecialChar ~

\layout LaTeX


\backslash 
pagebreak
\layout Standard


\begin_inset LatexCommand \tableofcontents{}

\end_inset 


\layout LaTeX


\backslash 
pagebreak
\layout Section

Introduction
\layout Standard

This document is about how IPFC generates, manages and transfers its messages.
 We describe both the requirements for an IPFC compatible protocol (both
 message passing and file contents) as well as the expected behavior of
 the IPFC running infrastructure.
\layout Standard

The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD
 NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted
 as described in IETF RFC2119 (http://ietf.org/rfc/rfc2119.txt).
\layout Subsection

IPFC protocol 
\layout Standard

The IPFC protocol transports IPFC messages (XML files) across the different
 zones in the framework.
\layout Description

Pusher A pusher is anything that can talk using the IPFC message passing
 protocol.
\layout Description

DR-Server A DR-Server is a HTTP server compliant to the IPFC HTTP protocol.
 A DR-Server MUST implement all the IPFC HTTP protocol REQUIRED functionality.
\layout Description

Agent An agent uses a wrapper to communicate in the IPFC framework.
 An agent can be anything from an embedded device to a mainframe.
\layout Description

Fetcher A fetcher is anything that can talk (using the IPFC message passing
 protocol).
 A fetcher MUST implement all the fetcher REQUIRED functionality.
\layout Subsection

IPFC data format
\layout Standard

The basic IPFC message data format is an XML file that is sent through the
 IPFC infrastructure using the IPFC protocol.
\layout Section

IPFC HTTP protocol
\layout Subsection

Protocol description
\layout Standard

The IPFC protocol is based upon the HTTP protocol.
 The IPFC protocol REQUIRES to support these methods : 
\layout Itemize

GET
\layout Itemize

PUT
\layout Itemize

HEAD
\layout Itemize

POST
\layout Standard

The IPFC protocol REQUIRES to support HTTP/0.9 and HTTP/1.0.
 For efficiency (eg.
 the use of persistent connections), the IPFC protocol SHOULD use HTTP/1.1.
 The IPFC protocol SHOULD be encapsulated in TLS/SSLv3 connections for encryptio
n and authentication of the data exchange.
\layout Subsection

Wrapper functionality
\layout Standard

A Wrapper is a software or an embedded software that collects/sends data
 from/to the Agent.
 An IPFC HTTP Wrapper MUST implement the REQUIRED IPFC HTTP protocol functionali
ty.
 
\layout Standard

The Wrapper MUST initiate the connection only against the DR-Server.
 The connection MUST be TCP oriented connection and MUST use HTTP as Transport.
 The Wrapper SHOULD encapsulate his HTTP connection in a TLS/SSL connection.
 The Wrapper SHOULD use HTTP keep-alive when using HTTP/1.1 for performance
 reasons.
\layout Subsubsection

Wrapper exchange 
\layout Standard

The Wrapper is only initiating TCP connections to the DR-Server by using
 the IPFC HTTP protocol.
\layout Subsubsection

Wrapper pushing
\layout Standard

The pushing for the wrapper works as follows :
\layout Standard

The wrapper checks if the file it wants to push to the dr-server is not
 already there.
 It does this by requesting the corresponding 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file.
 If this file is already on the dr-server, the Wrapper will stop the pushing
 of the file and will move it into a 
\begin_inset Quotes eld
\end_inset 

bad queue
\begin_inset Quotes erd
\end_inset 

.
 This is unlikely to happen but it avoids the wrapper trying to send a duplicate
 file all the time to the dr-server.
\layout Standard

If the 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file does not yet exist, the wrapper will PUT the file and the corresponding
 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file.
 The presence of the 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file indicates that the main file was completely transferred.
\layout Paragraph

Example of pushing an event
\layout LyX-Code

W : Wrapper
\layout LyX-Code

D : Dr-Server
\layout Subparagraph

From Wrapper to DR-Server (W->D)
\layout Standard

In this example, the protocol HTTP/1.1 is in use with keepalive and without
 using TLS/SSL encapsulation.
 The connection is initiated by the wrapper.
 The wrapper will issue a GET command to verify the non-existence of the
 same 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file.
 
\layout LyX-Code

GET /ipfc/tournesol/events/2001112200203306251.events.ok HTTP/1.1
\layout LyX-Code

Keep-Alive: 300 
\layout LyX-Code

Connection: Keep-Alive 
\layout LyX-Code

Host: 192.168.1.1:80 
\layout LyX-Code

Authorization: Basic YWxlSiissskyUE= 
\layout LyX-Code

From: Wrapper name: 
\layout LyX-Code

User-Agent: IPFC genwarp cvs ver 
\layout LyX-Code

Content-Length: 1989
\layout Subparagraph

From D to W
\layout Standard

The DR-Server replies that the file does not exist.
\layout LyX-Code

HTTP/1.1 404 Not Found 
\layout LyX-Code

Date: Fri, 23 Nov 2001 16:16:01 GMT
\layout LyX-Code

Server: Apache/1.3.20 (Unix) mod_gzip/1.3.19.1a 
\layout LyX-Code

\layout Subparagraph

From W to D
\layout Standard

The file is transferred to the DR-Server by using the PUT method.
 
\layout LyX-Code

PUT /ipfc/tournesol/events/2001112200203306251.events HTTP/1.1
\layout LyX-Code

Keep-Alive: 300 
\layout LyX-Code

Connection: Keep-Alive 
\layout LyX-Code

Host: 192.168.1.1:80 
\layout LyX-Code

Authorization: Basic YWxlSiissskyUE= 
\layout LyX-Code

From: Wrapper name: 
\layout LyX-Code

User-Agent: IPFC genwarp cvs ver 
\layout LyX-Code

Content-Length: 1989
\layout LyX-Code

\layout LyX-Code

<?xml version='1' standalone='yes'?>
\layout LyX-Code

<ipfc>
\layout LyX-Code

...
\layout LyX-Code

</ipfc>
\layout Subparagraph

From D to W
\layout LyX-Code

HTTP/1.1 200 OK
\layout LyX-Code

Date: Fri, 23 Nov 2001 16:16:01 GMT 
\layout LyX-Code

Server: Apache/1.3.20 (Unix) mod_gzip/1.3.19.1a 
\layout LyX-Code

Keep-Alive: timeout=15, max=100 
\layout LyX-Code

Connection: Keep-Alive 
\layout LyX-Code

Transfer-Encoding: chunked 
\layout LyX-Code

Content-Type: text/plain 
\layout LyX-Code

4d 
\layout LyX-Code

 
\layout LyX-Code

<HTML>File /ipfc/tournesol/events/2001112200203306251.events created.</HTML>
 
\layout LyX-Code

0
\layout Subparagraph

From W to D
\layout Standard

If the events file is correctly created on the DR-Server, the wrapper MUST
 create an 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 file.
\layout LyX-Code

PUT /ipfc/tournesol/events/2001112200203306251.events.ok HTTP/1.1 
\layout LyX-Code

Host: 195.207.52.190:44000 
\layout LyX-Code

Authorization: Basic YWxleDp2Sezrzjbng= 
\layout LyX-Code

From: Wrapper name: 
\layout LyX-Code

User-Agent: IPFC genwarp cvs ver
\layout Subparagraph

From D to W
\layout LyX-Code

HTTP/1.1 200 OK 
\layout LyX-Code

Date: Fri, 23 Nov 2001 16:16:02 GMT 
\layout LyX-Code

Server: Apache/1.3.20 (Unix) mod_gzip/1.3.19.1a Transfer-Encoding: chunked 
\layout LyX-Code

Content-Type: text/plain 
\layout LyX-Code

50  
\layout LyX-Code

<HTML>File /ipfc/tournesol/events/2001112200203306251.events.ok created.</HTML>
 
\layout LyX-Code

0 
\layout Standard

Only now the wrapper can remove the corresponding file from its queue.
\layout Subsubsection

Wrapper fetching
\layout Standard

The fetching functionnality for the Wrapper is in use when the Wrapper can
 get policy or configuration files.
 The fetching is working like that :
\layout Itemize

Get a directory listing of the specified directory (policy,config...)
\layout Itemize

Get all policy files with corresponding 
\begin_inset Quotes eld
\end_inset 

.ok files
\begin_inset Quotes erd
\end_inset 

 and NOT with 
\begin_inset Quotes eld
\end_inset 

.get
\begin_inset Quotes erd
\end_inset 

 files
\layout Itemize

Create a 
\begin_inset Quotes eld
\end_inset 

.get
\begin_inset Quotes erd
\end_inset 

 files with all policy correctly transfered
\layout Itemize

Process policy files and Create 
\begin_inset Quotes eld
\end_inset 

.result
\begin_inset Quotes erd
\end_inset 

 files with the results with a corresponding 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 files
\layout Subsection

db-wrapnet functionalities
\layout Standard

The fetcher has the purpose to get all the files from the dr-server that
 will be processed by the db-backend.
 
\layout Subsubsection

db-wrapnet exchange 
\layout Standard

The Fetcher is only initiating TCP connection to the DR-Server by using
 the IPFC protocol functionnalities.
\layout Subsubsection

db-wrapnet fetching
\layout Standard

The fetching works as follows :
\layout Itemize

Get a directory listing of the specified directory for the specified agent
\layout Itemize

Get all files for these directories with corresponding 
\begin_inset Quotes eld
\end_inset 

.ok
\begin_inset Quotes erd
\end_inset 

 files
\layout Itemize

Create a 
\begin_inset Quotes eld
\end_inset 

.processed
\begin_inset Quotes erd
\end_inset 

 file in the corresponding processed directory (there can be multiple processed
 directories if multiple backends are present : where the file is written
 is defined in the db-wrapnet configuration) for the specified agent
\layout Section
\pagebreak_top 
IPFC data format
\layout Standard

The IPFC data format is based on XML.
 This offers the advantage that we can use standard parsers and tools to
 process information from the XML files.
 Another advantage of XML is the possibility to be backward compatible in
 the expression of the document, by carefully choosing the used XML tags.
\layout Subsection

IPFC XML description
\layout Itemize


\begin_inset LatexCommand \label{<ipfc>}

\end_inset 


\series bold 
<ipfc>
\series default 
 
\begin_deeper 
\layout Standard

used to start a new IPFC xml document
\layout Itemize


\series bold 
version=
\begin_inset Quotes erd
\end_inset 

1
\begin_inset Quotes erd
\end_inset 


\series default 
 
\begin_deeper 
\layout Standard

the version of the file format (always 
\begin_inset Quotes eld
\end_inset 

1
\begin_inset Quotes erd
\end_inset 

 for now)
\end_deeper 
\layout Itemize


\series bold 
type=
\emph on 

\begin_inset Quotes erd
\end_inset 

type
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

gives the type of the entry.
 This is not used in the db-backend but may be used by low-level tools wishing
 to read directly from the dr-server to monitor eg.
 the file upload process.
 Typical types include 
\emph on 
status
\emph default 
, 
\emph on 
events
\emph default 
, 
\emph on 
alerts
\end_deeper 
\layout Itemize


\begin_inset LatexCommand \label{<data>}

\end_inset 


\series bold 
<data>
\begin_deeper 
\layout Standard

used to encapsulate the 
\begin_inset Quotes eld
\end_inset 

useful
\begin_inset Quotes erd
\end_inset 

 data which must be put in the db-backend.
 Multiple 
\emph on 
<data>
\emph default 
 blocks may exist in one file.
\layout Itemize


\series bold 
type=
\emph on 

\begin_inset Quotes erd
\end_inset 

type
\begin_inset Quotes erd
\end_inset 


\emph default 
 
\begin_deeper 
\layout Standard

gives the type of the entry (see below for more detailed description)
\end_deeper 
\layout Itemize


\series bold 
subtype=
\emph on 

\begin_inset Quotes erd
\end_inset 

subtype
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

defines the subtype of the entry (see below for more detailed description)
\end_deeper 
\layout Itemize


\series bold 
version=
\emph on 

\begin_inset Quotes erd
\end_inset 

version
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

version of the entry (see below for more detailed description)
\end_deeper 
\layout Itemize


\series bold 
transport=
\emph on 

\begin_inset Quotes erd
\end_inset 

transport
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

name of the transport method used to encapsulate the actual data (eg.
 syslog-line, log-line, null, etc...)
\end_deeper 
\layout Itemize


\series bold 
\emph on 
<various\SpecialChar ~
tags>
\emph default 
 
\begin_deeper 
\layout Standard

encapsulates the actual content of the specified type (for some agent types)
\end_deeper 
\end_deeper 
\end_deeper 
\layout Itemize


\series bold 
<agent>
\begin_deeper 
\layout Standard

information about the agent sending the data entries
\layout Itemize


\series bold 
id=
\emph on 

\begin_inset Quotes erd
\end_inset 

id
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

agent_id - a number, required to identify the agent (unique over all agents
 managed by a single db-backend)
\end_deeper 
\layout Itemize


\series bold 
wrapperid=
\emph on 

\begin_inset Quotes erd
\end_inset 

wrapperid
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

a number identifying a sub-wrapper of a certain agent.
 This parameter is optional and defaults then to 
\emph on 
0
\emph default 
.
\end_deeper 
\layout Itemize


\series bold 
date=
\emph on 

\begin_inset Quotes erd
\end_inset 

timestamp
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

timestamp (style 
\begin_inset Quotes eld
\end_inset 

20001-12-17 13:48:42
\begin_inset Quotes erd
\end_inset 

) of the agent at the time of the XML file generation
\end_deeper 
\layout Itemize


\series bold 
generationid=
\emph on 

\begin_inset Quotes erd
\end_inset 

timestamp
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

a timestamp which identifies the start-up time of the wrapper
\end_deeper 
\layout Itemize


\series bold 
sequenceid=
\emph on 

\begin_inset Quotes erd
\end_inset 

sequenceid
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

numeric sequence number of this file
\end_deeper 
\layout Itemize


\series bold 
transacid=
\emph on 

\begin_inset Quotes erd
\end_inset 

transacid
\begin_inset Quotes erd
\end_inset 


\begin_deeper 
\layout Standard

a text giving a unique transaction number.
 This number is stored in the db-backend but not further used.
\end_deeper 
\end_deeper 
\layout Subsection

Defined 
\emph on 
<transport>
\emph default 
 types
\layout Standard

In IPFC v1.0.1, the following 
\emph on 
<transport>
\emph default 
 types are possible :
\layout Subsubsection

null
\layout Subsubsection

log-line
\layout Standard

In the log-line transport, the type/subtype dependent parsing function will
 be given line by line of the decoded 
\emph on 
<log-line
\emph default 
> entries available in the 
\emph on 
<data>
\emph default 
 block.
 The content of the 
\emph on 
<log-line
\emph default 
> entries is decoded using the 
\emph on 
format=
\begin_inset Quotes erd
\end_inset 

format
\begin_inset Quotes erd
\end_inset 


\emph default 
 attribute (which is optional).
\layout Subsubsection

syslog-line
\layout Standard

In the syslog-line transport, the type/subtype dependent parsing function
 will be given line by line of the parsed and decoded 
\emph on 
<syslog-line>
\emph default 
 entries available in the 
\emph on 
<data>
\emph default 
 block.
 The content of the 
\emph on 
<syslog-line
\emph default 
> entries is decoded using the 
\emph on 
format=
\begin_inset Quotes erd
\end_inset 

format
\begin_inset Quotes erd
\end_inset 


\emph default 
 attribute (which is optional).
\layout Standard

The syslog-line transport will partially parse the given syslog-line and
 fills in some (syslog-specific and generic) database fields.
 These can be overridden in the type/subtype dependent parsing function.
 The parsed and decoded 
\emph on 
<syslog-line> 
\emph default 
given to this function is the 
\begin_inset Quotes eld
\end_inset 

user message
\begin_inset Quotes erd
\end_inset 

 part of the syslog message (so all syslog-specific data is already chopped
 off).
 In case you want to parse the whole line yourself, it is recommended to
 use the log-line transport.
\layout Subsection

Defined 
\emph on 
<type><category>
\emph default 
 pairs
\layout Subsubsection

Apache : log/common-log
\layout Standard

<data type=
\begin_inset Quotes erd
\end_inset 

log
\begin_inset Quotes erd
\end_inset 

 subtype=
\begin_inset Quotes erd
\end_inset 

common-log
\begin_inset Quotes erd
\end_inset 

 version=
\begin_inset Quotes erd
\end_inset 

1
\begin_inset Quotes erd
\end_inset 

 transport=
\begin_inset Quotes erd
\end_inset 

log-line
\begin_inset Quotes erd
\end_inset 

>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<log-line> ....
 
\emph on 
apache common log (access_log) line ...

\emph default 
 </log-line>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<log-line> ....
 </log-line>
\newline 
</data>
\layout Subsubsection

Apache : log/error-log
\layout Standard

<data type=
\begin_inset Quotes erd
\end_inset 

log
\begin_inset Quotes erd
\end_inset 

 subtype=
\begin_inset Quotes erd
\end_inset 

error-log
\begin_inset Quotes erd
\end_inset 

 version=
\begin_inset Quotes erd
\end_inset 

1
\begin_inset Quotes erd
\end_inset 

 transport=
\begin_inset Quotes erd
\end_inset 

log-line
\begin_inset Quotes erd
\end_inset 

>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<log-line> ....
 
\emph on 
apache error_log line ...

\emph default 
 </log-line>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<log-line> ....
 </log-line>
\newline 
</data>
\layout Subsubsection

Syslog : log/syslog-line
\layout Standard

<data type=
\begin_inset Quotes erd
\end_inset 

log
\begin_inset Quotes erd
\end_inset 

 subtype=
\begin_inset Quotes erd
\end_inset 

syslog-line
\begin_inset Quotes erd
\end_inset 

 version=
\begin_inset Quotes erd
\end_inset 

1
\begin_inset Quotes erd
\end_inset 

 transport=
\begin_inset Quotes erd
\end_inset 

syslog-line
\begin_inset Quotes erd
\end_inset 

>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<syslog-line> ....
 
\emph on 
syslog line ...

\emph default 
 </syslog-line>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<syslog-line> ....
 </syslog-line>
\newline 
</data>
\layout Subsubsection

Netfilter : log/log
\layout Standard

<data type=
\begin_inset Quotes erd
\end_inset 

log
\begin_inset Quotes erd
\end_inset 

 subtype=
\begin_inset Quotes erd
\end_inset 

log
\begin_inset Quotes erd
\end_inset 

 version=
\begin_inset Quotes erd
\end_inset 

1
\begin_inset Quotes erd
\end_inset 

 transport=
\begin_inset Quotes erd
\end_inset 

syslog-line
\begin_inset Quotes erd
\end_inset 

>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<syslog-line> ....
 
\emph on 
syslog line containing netfilter logs ...

\emph default 
 </syslog-line>
\newline 
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
\SpecialChar ~
<syslog-line> ....
 </syslog-line>
\newline 
</data>
\layout Subsection

IPFC example files 
\layout Standard

IPFC example files can be found in the 
\emph on 
xml-examples
\emph default 
 directory.
\layout Section

Additional information
\the_end
