Ganymede Release 1.0.5
July 27, 2001
THINGSTODO
---------------------------------------------

0) Database rework to support full history tracking

Right now, the Ganymede server maintains in its database
only the current state of the server's data.  No historical
information is retained in a form which would allow the
builder tasks to easily handle the reporting of
differentials to external processes.

That is, we would like to have Ganymede be able to write out a file
that says something like 'since the last time this builder process was
run, 3 new users were created, one user was added to group labusers, 2
groups were deleted, and one user was renamed', rather than simply
dumping out the current state of the database and just expecting that
to be bulk-loaded into the environment.  Things like LDAP and NT
maintenance could be supported a lot better if Ganymede was able to
provide limited delta records rather than forcing a complete reload
from the Ganymede server's state.

1) DNS Support

The biggest point to make about Ganymede right now is that it is as
much a framework for you to build management software with as it is a
finished product.  In particular, while the gasharl, linux, and bsd
schemas are in decent shape, you have to either already be using GASH
or be willing to settle for simple user/group management if you aren't
willing to do some work to develop a schema that is useful for you.

What is really needed is for someone to develop a generic DNS schema
kit that includes a good general parser for DNS files.  The gasharl
schema is designed to support a single comprehensive NIS domain and a
single DNS domain broken into Class-C sized subnets, but it is
designed to work with files generated/maintained by GASH.

Bootstrapping a DNS management system from zone files needs to be
supported, but Ganymede's objects and links model makes this process
of doing this a bit unobvious.  The gasharl schema uses system
information where all the interfaces and system information for
individual systems is grouped together in objects.  For our network,
this information was collated and assembled by hand.  Hopefully
there's a good way of avoiding that work.

Right now the gasharl schema kit is the most fully developed and
tested schema, but the gasharl schema kit does not include a full
and working NIS/DNS build system.

We are currently developing enhanced DNS support through the use of an
XML layer to replace the old gash hosts_info file.  We intend to
support DNS fairly comprehensively with this software, but it is not
yet ready for release.

2) Scalability issues

Right now, the Ganymede server loads all data managed into memory.
This is reasonably efficient with a good generational garbage
collector, but it's possible that using a random access binary file
for the database, when combined with an in-memory cache system using
the JDK 1.2 weak references API could give better scalability.  Memory
is cheap these days, so the Ganymede design isn't too bad for our
purposes and size, but it might be good to experiment with the basic
engine to see whether things can be made more scalable with 1.2.

In addition, the client has some scalability issues.  In particular,
there are places where the client will generate pop-up choice lists
with hundreds if not thousands of choices in them.  This is not
workable from a user interface perspective, and may be a real burden
on the client's memory.  For real scalability it might be better to
require the user to manually type in items when the lists get too big.

3) Documentation/Help

The Ganymede client includes support for a basic HTML help system, but
we don't have the HTML help web written.  Given our resources and
priorities, it isn't clear that we'll be able to do this ourselves
anytime soon, but it really should be done.

4) Internationalization

It would be a really good thing for the client and server to be
internationalized.  After getting the Ganymede announce posted in
Linux Weekly News and seeing all of the hits from Japan, Italy,
Austria, Russia, China, the Netherlands, France, Spain, Portugal,
etc., my fantasy that internationalization was something I could just
sort of pretend I'd never heard of seems to be evaporating.  I don't
think I'll be able to dedicate time to it anytime in the foreseeable
future, but someone is going to have to at some point.

5) Two phase commit for client access

The Ganymede server internally uses a two-phase commit protocol for
handling its transactions, but it does not provide a two-phase
interface for clients that may be coordinating transaction commits for
an external system.  As a result, while Ganymede can be used to master
cross-database transaction commits using the DBEditObject
commitPhase1() and commitPhase2() methods, it doesn't support a client
attempting to verify transactional consistency without performing an
actual commit.

No one has asked about this yet, but at some point I may need to
expand the client API to allow some sort of 'verify ok to commit'
method call which, if it returns a successful result, will guarantee
that a later commit would proceed to completion successfully.  This
will involve a good bit of reworking of the server's internal commit
operations, and there may be issues of deadlock and timeouts to be
considered as well.  At present, transactions lock all portions of the
database involved in the transaction while the commit logic is being
processed, but if there might be an indefinite delay between phases 1
and 2 of an externally-accessible two phase protocol, we have to
either risk unlocking the server and then hoping we can re-lock the
server in a timely fashion later, or risk leaving those portions of
the server locked until such time as a phase 2 order arrives.

I'm not sure how folks handle this problem, generally.

6) Integration of CVS into the backend

Right now, Ganymede is doing a lot of stuff in the server to handle
archiving of emitted files.  Things like the automatic saving of
backup copies of a file when a file is emitted by a builder task, and
the nightly zipping of the day's backup files.

Really, this would be better done by having the backend scripts assume
the presence of CVS, and to do CVS commits every time new files are
written out.  This would allow for much more efficient storage of the
emitted files, and would allow for easy retrieval of the emitted files
for any given moment in time.

Schema kits should be developed in this direction.

---

Jonathan Abbey, jonabbey@arlut.utexas.edu
