----------------------------------------------------------------------
|                             A B S E N T                            |
|             http://www.research.att.com/projects/absent            |
|                                                                    |
|                                by                                  |
|                                                                    |
|            Christian Gilmore, David Kormann, and Avi Rubin         |
----------------------------------------------------------------------

Welcome to the first public release of the Absent system.  Absent
provides traveling users with the ability to browse Intranet websites
without compromising security and without any special browser features
beyond SSL and HTML.

READ THIS ENTIRE FILE BEFORE EVEN ATTEMPTING TO COMPILE ABSENT.
Trust us, if you don't read all the way through, you won't get absent
compiled and you'll just be upset and frustrated and want to kill the
absent developers.  Really.  Absent's build process is currently
horribly complex and broken, and if you don't follow closely, you'll
go barking mad.

Before going any further, you might also want to read the absent
techreport, which describes the system and should give you a feel for how
it works.  The paper is in the same directory as this INSTALL file,
and is named 'absent-TR.ps'

SUPPORTED PLATFORMS
Absent has been built on Solaris 2.5.1 and 2.6; some effort has been
made to make it build on linux (2.2.3, libc 2.0.7), but we have done
no testing with linux, and we can't guarantee it'll work.

We've had the most experience running it on solaris machines, though
there's no reason it shouldn't run on anything its components compile
on.  If you're building absent on linux, you need a functional
/dev/random; truerand doesn't function correctly on linux.  truerand
has also been reputed to have problems on some Irix variants.  If you
find that gen_key hangs forever or produces predicatable output, it's
truerand.

THIS IS AN ALPHA RELEASE
As you'll probably guess from the complex build process, this is a
very early release of the absent system; it has known bugs and is very
difficult to compile and install.  WE CANNOT ACCEPT RESPONSIBLITY FOR
SECURITY PROBLEMS ARISING FROM THIS OR ANY RELEASE OF THE ABSENT
SYSTEM.  Be aware that if you screw up the installation, the
consequences for the security of your internal network could be dire.
If you're not comfortable editing Perl scripts and installing a
complex and buggy piece of software, this release isn't for you.  Be
sure to read the BUGS section at the end of this document for more
information on failures to expect from this version of the absent
system.  All that being said, we've been running the system at AT&T
for about a year as of this release, and we're reasonably happy with
the stability and function of the system.


INSTALLATION
1. CONFIGURING YOUR NETWORK
The absent system requires two hosts, one inside, and one outside a
packet-filtering firewall (an application-layer gateway would work as
well, but we do not provide the appropriate proxy.  you'd need to
construct that yourself).  We refer to the inside host as "Pushweb"
and the outside host (confusingly enough) as "Absent".  The Pushweb
host runs a modified version of apache and serves as a web proxy.  It
must have access to any internal sites you wish to serve with absent.
Moreover, the Pushweb host must be allowed to establish TCP
connections across the firewall.  If you wish, you may restrict
outgoing TCP across your firewall so that only the ports used by the
absent system are permitted.  By default, these ports are 4431 and a
configurable number of ports above and including 4500 (by default,
ports 4500-4756 are used).  All of these values are configurable at
runtime.  Port 443 of absent must be accessible from the internet.
Any other ports on absent may (and probably should) be firewalled off
from the internet.  We recommend that no other processes be run on
either the absent or Pushweb host.

Most of the compiling work you'll be doing will be for software which
will run on Pushweb, the inside host.  You'll probably find it
simplest to do all your compiling on that particular host, then copy
the absent daemon to the absent host (we're going to assume that
absent and Pushweb have the same architecture and operating system;
while absent _may_ work across hardware and operating systems, it has
never been tested).

You'll also want to decide on users to run the Pushweb and Absent
daemons.  We STRONGLY recommend that these users be unprivileged in
every respect: untypable passwords and unable to write to any file
except (in the case of Pushweb) the few runtime files necessary to the
operation of pushweb (specifically, the OPIE locks directory and the
user and logs directories).  At present, neither Pushweb nor Absent is
really designed to run in a chroot()ed environment.  We will be fixing
this; in the meantime, it may still be possible to make it work, but
we haven't tried, and we suspect you'll run into problems.  If you
manage to make it work, please let us know.


2. UPDATING YOUR PERL INSTALLATION
Before beginning to build absent, install Perl 5 must be installed on
the host which will serve as Pushweb, the internal proxy.  To the
perl installation on this machine, you'll need to add some modules:
 - Digest::MD5 (for libwww and the hmac_md5 function)
 - HTML::Parser (for libwww)
 - MIME::Base64 (for libwww)
 - URI (for libwww)
 - CGI (for libwww)
 - Data::Dumper (for libnet)
 - libnet (for libwww; in CPAN/modules/by-module/Net at CPAN archives)
 - libwww (in CPAN/modules/by-module/WWW)

Note that we don't specify version numbers here; if you've already got
a fairly recent version of these packages installed on the machine
you'll be using as Pushweb, you can use those packages.  Installing
these packages will overwrite any existing installation of them; if
you're using this perl installation for other applications (we don't
recommend that, but hey, we're realistic), be aware that updating
these packages could affect those applications.

All of these modules are available from your nearest CPAN mirror.  If
you've got the CPAN perl module you can fetch libwww, and it'll get
most of the rest.  We're not going to discuss how to build these here;
see each module's documentation for help on building it.  It is only
necessary to have these modules installed in Pushweb's Perl
installation; no Perl installation is required on the host Absent.

Fortunately, even if you don't have the CPAN module, installing the
perl modules is extremely straightforward: for each module, grab the
module source from CPAN, untar it, cd into the source tree and type

  perl Makefile.PL
  make all
  make install

all of the modules should build in a fairly straightforward fashion;
the only caveat is to be absolutely sure they're installing into the
perl you'll be using on pushweb.


3. SOFTWARE YOU'LL NEED
Once you've added these modules to your perl installation, you'll want
to fetch the software you'll need to build the pushweb daemon:
- Opie 2.32 (from ftp://ftp.gbnet.net/pub/security/nrl-opie)
- Apache 1.3.9 (from http://www.apache.org/)
- mod_perl 1.2.1 (from http://perl.apache.org)
Optionally (see below):
 - OpenSSL 0.9.4 (from http://www.openssl.org)
 - apache-ssl (from http://www.apache-ssl.org)
(the version numbers here are the versions we've tested with.  you may
have trouble with other versions, particularly in the case of apache.)

you should unpack each of these packages in the same directory as this
INSTALL file.  after you're done unpacking them, you should have
something that looks appropximately like this:

$ ls -la /usr/local/src/absent

  drwxr-xr-x   9 davek    davek        1024 Jun 15 19:20 .
  drwxrwxr-x   5 davek    davek        1024 Jun 15 17:54 ..
  drwxrwxr-x   5 davek    davek       17700 Jun 15 17:54 INSTALL
  drwxrwxr-x   5 davek    davek        1110 Jun 15 17:54 README
  drwxr-xr-x   9 davek    davek        1024 Jun 15 17:55 apache_1.3.9
  drwxr-xr-x  24 davek    davek        1024 Jun 15 19:20 mod_perl-1.21
  drwxr-xr-x  21 davek    davek        1024 Jun 15 17:57 openssl-0.9.4
  drwxr-xr-x   4 davek    davek        1024 Jun 15 17:57 opie-2.32
  drwxr-xr-x   2 davek    davek        1024 Jun 15 17:54 src

The absent makefiles depend on this organization right now; if you
rearrange the directory tree, you'll need to edit makefiles (we'll
point out where later).

3a. OPIE

BE VERY CAREFUL WHEN INSTALLING OPIE
OPIE is designed to replace your machine's normal login process.
To this end, it'll happily replace /bin/login.  if you don't want that
to happen, be sure you don't use 'make install' or 'make
install-server'.

the first step you'll want to take is to build OPIE.  OPIE comes with
a configure script and should be fairly straightforward to compile.
if you decide to install the OPIE software globally at this point, be
sure you do it on the machine which will be Pushweb.  It is not
necessary to have OPIE installed on the machine which will be Absent,
and opie is not required to build the daemon which runs on that host.

If you're planning to use the perl user-registration function, you'll
need to specify the --enable-insecure-override option when running
configure, like this:
	
	./configure --prefix=/usr/local/opie-2.32 --enable-insecure-override
		
In fact, it's not an entirely bad idea to set this option regardless
of whether you use the user registration scripts, as it makes OPIE a
lot easier to work with.

Before compiling OPIE, edit the Makefile generated by Configure and
search for the KEY_FILE macro.  By default, this is set to
"/etc/opiekeys".  Pushweb will be using the opiekeys file to store
challenge information about users.  You should pick a directory on the
pushweb server in which to store this file, make a note of the
directory, and set the KEY_FILE macro in the Makefile to reflect
this.  You may also want to change the value of LOCK_DIR; the OPIE
makefile puts this under /etc by default, which may not be appropriate
for your site.  Pick someplace the Pushweb user will be able to write
to.

OPIE will try to install some files (such as opiekey) setuid root;
absent doesn't require that these files be root-owned, only that the
absent user be able to read and write the OPIE keys file.  You can
take away the setuid bits from these files after installing them.

3b. MOD_PERL
Install mod_perl into the perl and apache you'll be using on
pushweb.  To do this, cd into the mod_perl source tree and run the
following commands:

  perl Makefile.PL USE_APACI=1 EVERYTHING=1
  (answer 'y' to the first question, 'n' to the second)
  make all
  make install

Again, be absolutely sure the perl you're using to build mod_perl is
the one you'll be using on the pushweb host.

The USE_APACI part is _very_ important; apache won't build if you
don't include it.  If you leave off EVERYTHING=1 apache will build,
but pushweb won't work, which would really take the point out of this
whole grisly enterprise.

The mod_perl configuration script asks two questions; the first just
verifies that it guessed the right apache source tree; make sure that
the path shown leads to the apache source tree you'll be using for
pushweb.  The second question asks whether you want mod_perl to build
apache for you.  Answer 'n'.  You'll need to apply some patches first.

mod_perl both applies patches to your apache source tree and
installs a Perl module.  You should ensure that the Perl module is
being installed on the system which will be the Pushweb server, and
that the patches apply to the source tree you'll be using for the
Pushweb apache installation.  

  3c. OPTIONAL STEP: SSL (READ THIS BEFORE DECIDING _NOT_ TO TAKE THE OPTION)
  The security of the absent system depends directly on the presence of
  an SSL connection between the browser and the Apache server which
  Pushweb uses.  However, SSL isn't, strictly speaking, necessary and
  pushweb can be built just fine without it.  it just won't be the
  slightest bit secure, and we cannot support this use of absent except
  as a means of dodging some byzantine regulations on the use of
  cryptography (without SSL, there should be no export or cryptography
  issues related to the use of absent).  If you decide to use OpenSSL,
  build it now.  again, if you decide to install the software 
  globally at this point, be sure you're doing it on pushweb.  Next, you
  should apply the SSL patches to the apache source.  see the README.SSL
  file, included in the apache-ssl package and located at the root of
  the apache source tree, for information on how to do this.  You'll
  also need to get a certificate issued for this server in order to
  use SSL.  You should request that the certificate be issued for the
  host on which you'll be running the absent (outside) daemon, as it
  is that host with which clients will connect (you'll be engaging in
  a small white lie, as the certificate will actually be installed
  on pushweb.  we won't tell if you don't).  This, unfortunately, is
  about all we can tell you about installing SSL due to US export
  laws.


4. BUILDING AND INSTALLING THE DAEMONS
Absentd provides reverse-proxy services and runs on the external host,
absent.  To build it (and some supporting files which will be used to
build pushweb) first cd into the src directory and edit the Makefile
you find there.  Check the values of the CC, PERL, PATCH, LIBRAND,
LOCINC, APACHE, OPIEDIR, INCOPIE, LIBOPIE, and OLIBS variables, and
set them appropriately.  

There are also two directories which are, at present, hardcoded into
the Pushweb binary; you probably want to set these now as the files
which use them will be compiled as part of absentd (though the paths
aren't used by absentd).  The macros for these paths are MASTER_KEY
and USER_DIR.  MASTER_KEY is a filename; the file will contain a key
which will be used by Pushweb to generate subkeys to authenticate
URLs.  You should ensure that this file does not become available to
intruders.  The file must be readable, but need not be writable, by
the user who will run the Pushweb daemon.  USER_DIR is the directory
where Pushweb will store generated keys for users; this directory
should be writable by the user who will run the Pushweb daemon.  For
this reason, we recommend you resist the urge to consolidate your
configuration files (such as the MASTER_KEY, pushweb configuration
file, and opiekeys files) in the USER_DIR directory.

absentd does all its logging through syslog; by default, the facility
used is LOCAL5; if you'd rather it use some other facility, edit
include/absentd.h and fix the line

#define ABSENTD_LOGFAC LOG_LOCAL5

to specify your preferred logging facility.

Once you've set things up appropriately, run

	make absentd

This should get you an absentd binary.  If you're compiling on
pushweb or some other host, you can copy this file, along
with the sample absentd.conf file you'll find in the src/conf directory,
to the absent host now.

5. BUILDING AND INSTALLING PUSHWEB
[ Pushweb shares some code with absent; if you're building absentd and
pushweb on different architectures, you should do a 'make clean' at
this point, and make sure the contents of the Makefile are sane for
the architecture on which you're building Pushweb. ]

Pushweb consists of an apache module, a few perl scripts, and some
patches to apache's http_main.c.  The pushweb module's makefile will
attempt to install these patches.  If you haven't got a version of
patch which works with the context diff supplied, you can either get
and build GNU patch (from
ftp://gnudist.gnu.org/pub/gnu/patch-2.5.tar.gz) or apply the diff by
hand (it's a fairly simple diff, mostly a few added lines).  if you DO
have a version of patch, edit src/Makefile to reflect the location of
your patch program.

You'll want to pick an install location; for the purposes of this
document, we'll use /usr/local/pushweb, but you can really put things
anywhere you like.  Just be consistent.

The first step is to edit the perl scripts to reflect your install
location.  The perl scripts are located in src/mod_pushweb.  There are
two scripts here you'll need to edit: AbsentProxyRerwite.pm and
Response_Pages.pm.  AbsentProxyRewrite is the perl module which
performs URL rewriting on pushweb.  You should search for the string
"XXX EDIT ME XXX" in this file, and read the instructions you find
there.  Everything you need to edit is in the first chunk of the file.
You should pay particular attention to the line

use lib qw(/usr/local/pushweb/lib);

and the definitions of the variables ABSENT_URL and INTERNAL_SITES.
if these aren't set correctly, absent won't work.

Next, edit the file ResponsePages.pm, found in the same place as
AbsentProxyRewrite.pm.  ResponsePages.pm is just a set of variables
containing chunks of HTML for the static pages used by absent to
generate, for example, the login page.  Absent will function correctly
if you don't edit the contents of this file, but it'll look very
strange, and will contain references to AT&T Labs - Research.  Read
the instructions at the top of the file for information on how the
various variables are used.

Your next step, and probably the most difficult in this process, is
building apache.

The pushweb daemon needs to modify the apache server and add an apache
module.  To add the module to apache, cd into the src directory (the
absent src directory, not the apache src directory), and type 

	make pushweb

Note that this doesn't patch or compile apache, it just makes sure a
few necessary files are built, and copies the mod_pushweb tree.  After
this is done, you'll want to run "configure" at the root of the apache
source tree Minimally, you should run it like:

  ./configure --activate-module=src/modules/mod_pushweb/libmod_pushweb.a \
       	     --activate-module=src/modules/perl/libperl.a \
	     --enable-module=apache_ssl

but you'll probably want to also do things like set paths and
so on.  It'd be a good idea to disable all but the perl, pushweb, and
SSL modules; the security of the system depends on the security of the
pushweb service, and that's a lot easier to guarantee if you don't
have a lot of extensions to think about.  This command line should
produce a workable, but minimal, pushweb:

	 ./configure  --disable-module=actions \
	     --disable-module=alias --disable-module=asis \
	     --disable-module=auth --disable-module=autoindex \
	     --disable-module=cgi --disable-module=dir \
     	     --disable-module=env --disable-module=imap \
	     --disable-module=include --disable-module=mime \
	     --disable-module=negotiation --disable-module=setenvif \
	     --disable-module=status --disable-module=userdir \
             --activate-module=src/modules/mod_pushweb/libmod_pushweb.a \
             --activate-module=src/modules/perl/libperl.a \
	     --enable-module=apache_ssl

In other words, disable everything but the access, logging, and
apache_ssl modules, and add the perl and pushweb modules.  Pushweb
doesn't _require_ the access module to function, but it adds some nice
features you might want if there are chunks of the net which should be
allowed access to, or allowed access _from_, absent.

A tiny shell script which contains only the above configuration
command can be found in mod_pushweb/minimal_apache.sh.  It must be run
from the root of the apache source tree.

now cd to the mod_pushweb directory (in src/modules/mod_pushweb) and
type 

	make patch

this will attempt to apply a patch to http_main.c and fix up the makefiles.
if the patch fails, you may be able to apply it by hand, but unless
you're running a massively different version of apache, the patch
should apply fairly directly.

Finally cd back to the root of the apache tree and type "make".  After
your C compiler finishes grinding, you should have an httpsd (or
httpd, if you're not using SSL).  This is the pushweb daemon.

To install the pushweb daemon, select a directory on the machine which
will be pushweb; for this example, we'll use /usr/local/pushweb, but
you can really put the software anywhere you're comfortable.  Besides
the modified apache daemon, there are a few perl scripts you'll need
to copy.  the directory structure below shows one way to install them;
put things wherever you're comfortable putting them:

/usr/local/pushweb/
  bin/
    httpsd (from apache_1.3.9/src)
  sbin/
    gcache (from apache_1.3.9/src/modules/ssl, used by apache_ssl)
  etc/ (this tree should be readable only by the pushweb user)
    keys/ (this directory must be readable by the user running pushweb)
      master_key (generated by gen_key.  see below.)
      MACkey (generated by gen_key, copied to absent host)
      opiekeys (for OPIE; you should put this file wherever you
                set the KEY_FILE macro when building OPIE)
      (you might want to drop your SSL certificate and key here
       as well)
    users/ (this directory must be writable by the user running pushweb)
      ... user session keys will be stored here.
    cookies/ (this directory must be writable by the user running pushweb)
      ... cookies sent by remote web servers will be stored here
  conf/
    pushweb.conf (from src/conf)
    mime.types  (from apache_1.3.9/conf, required by apache)
  lib/
    Otp/
      (copy the entire Otp/ tree from
         apache-1.3.9/src/modules/mod_pushweb here.  it's not actually
         all necessary, but this is the simplest way to install it.)
    AbsentProxyRewrite.pm (from apache-1.3.9/src/modules/mod_pushweb)
    Response_Pages.pm (from apache-1.3.9/src/modules/mod_pushweb)
  logs/

Similarly, on the host which will run the absent daemon, you can
install things anywhere you'd like; we have a structure like:

/usr/local/absent
  bin/
    absentd (from src/)
  etc/
    MACkey (generated by gen_key, the same file as pushweb's MACkey)
    absentd.conf (from src/conf)
  log/
    absent_msgs (written by syslog; the file must exist for syslog to
                 write to it)

6. REGISTERING USERS, CONFIGURING AND RUNNING ABSENT
Before running absent for the first time, you must create two key
files.  The first, the file refered to as "MACkey" in the directory
structures above, will be used to authenticate control messages
between pushweb and absentd.  To create this file, run the program
gen_key, found in the src directory, like this:

./gen_key MACkey

gen_key takes a while to run.  Be patient.  The argument to the
program is a filename.  Call it whatever you want, and copy it to both
hosts, as shown above.  Next, you'll need to create a master key, from
which user session keys will be generated.  This is done by running
gen_key again, this time with a second argument:

./gen_key master_key 1

the second argument is a counter; absent generates new user keys by
encrypting the master key and an increment of this counter.  Put the
master key somewhere pushweb (and only pushweb, preferably) can get
to.

Before a user can log in to the absent system, they must be
registered; the registration process involves creating an OPIE key for
them, and placing them in the OPIE password file.  How you go about
doing this will depend on your local site policy and where you
installed OPIE.  We've included an example registration CGI script
which adds users to an OPIE database and sets other absent-related
information, such as their personal home page.  Look in the
"tools/" subdirectory in the absent source tree for this
program.  There's a README file in that directory which describes its
contents.  This script is simply a wrapper around the 'Otp' perl module
which you copied as part of the pushweb installation process; if you
decide to use the registration script, you'll need to copy that
module to any host which uses it, and figure some way to transfer
the opiekeys file between whatever host performs registration and the
pushweb host, if they're different.

On the host which will run absent, you should ensure that your syslog
daemon is logging the facility you've chosen for absent appropriately.
If you want to log absent daemon messages separately, you'll want to
add something like

local5.debug                    /usr/local/absent/log/absent_msgs

to /etc/syslog.conf (and restart syslog)

Absent and Pushweb have runtime configuration files which control some
aspects of their behavior.  You should inspect the examples you copied
from src/conf to the absent and pushweb installations, and edit the
values you find in them.  Each file has comments indicating what to
edit.

The absent and pushweb daemons may be started in any order; once you
start pushweb, it'll start trying to contact the host configured as
absent every 5 seconds until it gets a successful connection.  To
start pushweb, run the httpsd daemon you installed.  If you didn't
tell the apache configuration where you were installing it, you may
have to pass a configuration file name on the command line; something
like

/usr/local/pushweb/bin/httpsd -f /usr/local/pushweb/conf/pushweb.conf

should work.  Pushweb will print some messages to the console and
start trying to contact absent.  If it appears the daemon isn't
running, check the configured apache error log, and consider setting
the $debug variable to 1 in AbsentProxyRewrite.pm

To start absentd, run it like

/usr/local/absent/bin/absentd /usr/local/absent/etc/absentd.conf

absent will log most everything it's doing to syslog.

To test that your system's up and running, point a web browser at the
absent authentication page for a user you've registered.  If the value
of ABSENT_URL you used is https://absent.yoyodyne.com/ and you've
registered a user named "munchkin", this URL will be:

https://absent.yoyodyne.com/login=munchkin

You should be presented with a page with an OPIE challenge.  If not,
check the various logs.  If you're stuck, send mail to us and we'll
try to help.

Your users will need OPIE calculators in order to use their absent
accounts.  A (possibly somewhat dated) list of these can be found in
the file src/tools/calculators.html.  You may use this file to direct
your users to the appropriate calculator.

BUGS AND FUTURE WORK
- The most severe known bug in absent involves a loss of communications
between absent and pushweb.  If the system appears not to be
responding, try restarting both daemons.  We're hoping to have this
fixed soon.
- Absent's initial redirect after authentication isn't a syntactically
valid URI (it contains a second, unquoted ':').  Some browsers (most
notably IE 4.5 on the Macintosh) choke on this.  We'll fix it.
- pushweb may generate errors of the form:
  [error] (9)Bad file number: select: (listen)
  and:
  bind: Address already in use
this is due to a relatively harmless bug (related to the fact that at
startup, the control channel gets initialized multiple times).  we'll
fix it.
- Pushweb probably leaks some listener records across restarts.
- Obviously, the build process is a massive problem.  We'll improve
this soon.
- Pushweb's interface with apache is an awful hack.  We're
investigating replacements.
- At present, Pushweb only responds to PING messages, it never
initiates them.  This is broken, and may cause pushweb to be unaware
of a lost connection.  This will be fixed once we figure out a
sensible way to integrate the scheduling of pings with the apache
architecture.
- Some web servers don't return a Content-Type: header with documents.
Pushweb (well, apache, really) presently tags all documents like this
with the DefaultType in the apache configuration.  this isn't right,
and we've asked the apache folks what to do to fix it.
- Documentation is decidedly lacking.  We'll write man pages for
everything soon.
- Pushweb's page rewriting process makes frames look just terrible.
Every framed page gets a separate banner.  We have the design of a
fix.
