
Diablo V1.09

	Major bug in dnewslink found by Michael S. McMahon.  When dnewslink
	gets interrupted and refiles pending stream transactions back to 
	the batch file, it was not refiling the offset,size part!   A rerun 
	would thus result in the entire article spool file (encompassing many 
	articles) to be sent as a single article.  Ouch.

	Precommit caching added.  Diablo now generates and maintains a
	pcommit.cache file which it mmap's.  This file contains a
	static 4096-entry hash table.  Message-id's for check and ihave
	commands are entered into the hash table and timeout after 30 seconds.
	hash table collisions are simply overwritten.  During the time which
	a message-id is active, other check/ihave commands for the id will
	return a DUPlicate response code.   The precommit caching will get
	rid of 98% of the article collisions when you have several incoming
	feeds that are mostly synchronized with each other.  In this situation,
	the precommit caching will reduce your incoming network load by at 
	least a factor of 2 as well as reduce the disk write load.

	Three new statistical elements have been added: predup, posdup, and
	pcoll.  (predup and posdup were actually added in V1.08).  predup
	counts the number of history collisions that occur as of the
	beginning of a 'takethis' command.  posdup counts the number of history
	collisions that occur as of the article commit after a 'takethis'
	command, and pcoll counts the number of pcommit cache collisions
	which cause check/ihave to return a DUPlicated status.

	The random seed used to generate file names was not being re-randomized
	after a fork.  Oops!  It is now.

	Replaced stdio routines used for socket input by diablo with custom
	routines, removing the fileno(fi) = -1 problem and working around
	apparent stdio bugs in SunOs, Solaris, and IRIX.

	Article reception can deal with 8-bit-clean data now, as long as it
	is properly '.' escaped.  This includes removing the CR before an LF
	for storage and adding it back in for retransmission.

	Added 'd' option to dnntpspool.ctl, allowing one to specify a startup
	delay for the dnewslink's related to a particular site.

	statvfs fixes made for sun/solaris.  I got several diffs and chose
	the easiest one.  It may still not be entirely compatible with
	all sunos/solaris implementations.

Diablo V1.08	*** NOTE ** DHISTORY FILE RELOAD REQUIRED IF UPGRADING ***

	** HISTORY AND SPOOL FILE FORMAT CHANGE.  The format change requires
	   that you ensure that ALL of your diablo binaries are replaced.  You
	   must regenerate your dhistory file

	Major stability fixes to dhistory file operation... I have found that
	if the /news partition runs out of space or many processes are
	write()ing to dhistory in O_APPEND mode, that O_APPEND writes aren't
	as atomic as I thought they were.   dhistory appends are now
	serialized and *ACTIVELY* realigned if their alignment gets screwed
	up.  This should make the dhistory file much, much, much more robust.

	Major stability fixes to dqueue files.  Feeds no longer must be
	flushed on fork, and files are written in multiples of one line.
	If a write files, the file is truncated to remove the partial line
	in order to prevent corruption if a disk fills up.

	Stability fixes to article files... now does a sanity check (looks 
	for the \0 terminator for the article in a multi-article file) to
	catch corrupted files.

	didump -x does a better job filtering out bogus dhistory entries.

	Major, Major, Major changes in the spool file format, dhistory file
	format, and queue file format.  Read the section at the end of the
	INSTALL file for instructions on how to upgrade without blowing up
	your news server.

	    * spool files may now contain multiple articles separated by \0.
	      spool files are now named solely based on their gmtime and
	      iteration number.

	    * queue files include offset/length pairs to allow dnewslink to
	      access the new spool files.

	    * The dhistory file now has a proper header structure, and History
	      entries now contain two additional fields (offset, bytes) .. used
	      to store the offset,length information for article lookups.

	The dynamic pipe sizing has been removed from dnewslink.  It 
	doesn't work very well when feeds get behind and just slows
	things down more.

        dexpire is temporarily braindead... it now expires in straight FIFO
        fashion, so most of dexpire.ctl is no longer relevant (except for
        expirations of 0 which cause an article to be rejected)... I have
        a medium term plan to make expiration specifications work again, but
        it will be a few releases at the very least.


	diablo, dnewslink, and dexpire now use chdir() caching to reduce
	the path lengths for remove(), open(), etc... which is a big win
	for the kernel.

	diablo now records rejection statistics properly.  It previously
	did not count.

	diablo now records and returns the errno error string on fatal 
	errors.

	addition of 'delgroupany' command to dnewsfeeds file.  Similar
	to 'delgroup', except acts the same as INN's '@group'... i.e.
	if the group appears in the Newsgroups line at all, the article
	is rejected, even if there are other groups in the line that pass 
	the filter.

	addition of 'maxpath' command to dnewsfeeds file.  Allows you
	to filter feed based on the number of elements in the Path: 
	header.

	addition of the 'groupdef' command and 'groupref' command (see the
	sample dnewsfeeds file).  Rather then repeat the same group access
	list over and over again in each feed, you can now collect them
	together in one place and reference them from your feeds.

	statvfs support added for SUN.

Diablo V1.07	- ** NOTE ** YOU MUST READ THE 'INSTALL' FILE IF UPGRADING
		  TO THIS RELEASE FROM 1.05 OR LOWER !!!

	The diablo server now has the ability to bind to a specific host and/or
	ip address.

	diablo converts the reverse-lookup hostname to lowercase before 
	comparing against hosts in the diablo.hosts file.

	'nostream' option now available for dnntpspool.ctl for dnewslink runs.

	Diablo now rejects articles when the message-id in the article
	body does not match the message-id in the ihave or takethis header,
	and logs the mismatch.  Diablo previously stored the ihave/takethis
	message-id into the history file without checking it against the
	message-body.  If a mismatch occurs, the article is rejected.
	(should I do a 400+exit instead?)

	Misc small bug fixes to diablo

	Diablo now returns a 400 error code and exits if a file error occurs.
	It previously returned a 431 error code, which was incorrect.  400 
	isn't exactly correct either, since it requires diablo to exit.  We
	need another return code to allow the server to indicate a problem yet
	not exit (i.e. let the client decide to exit).

Diablo V1.06	- ** NOTE ** YOU MUST READ THE 'INSTALL' FILE IF UPGRADING
		  TO THIS RELEASE!!!!

	The default hash table size for dhistory will become USE_LARGE_HASH
	in this upcoming release.  If you wish to maintain a small hash table
	size (e.g. you are taking a partial feed), you must set the 
	USE_SMALL_HASH define to 1 in lib/vendor.h

	dnewslink will still attempt to send news if the (nntp) server 
	responds with code 201 rather then code 200.

	** It is suggested that the dclean admin script be run twice a week
	   rather then once a day due to it's load on the history file

	dnewslink now dynamically adjusts the size of the pipe for streaming
	feeds.  If dnewslink detects an inordinate number of 
	check/ok/takethis/reject collisions, it reduces the size of the
	pipeline.  The pipeline is also dynamically adjusted back up when
	the collisions subside.

	More /bin/sh script fixes in contrib/XMakefile... improper semicolons
	removed.

        Fixed bug in dexpire that caused it to loop endlessly without getting
	much work done.

	Added parallel-tasking option, -p# (as in -p4) to dexpire, causing it
	to fork N times in order to run multiple unlink()'s in parallel
	without the processes stepping over each other's feet.

Diablo V1.05

	Switched from whole-file locks on dhistotry to record locks.  Diablo
	was having lock starvation problems due to the large number of 
	processes accessing the history file.

	By using record locks, we lock a particular hash chain rather then
	the entire file.  This should make machines with a large number of 
	feeds more efficient.

	EFFICIENCY NOTICE:  We strongly suggest that you set the 
	USE_LARGE_HASH define to 1 in lib/vendor.h and follow the 
	instructions at the end of the INSTALL file to increase the
	size of the dhistory file hash table.  Doing so will yield a
	major increase in efficiency on machines with limited buffer caches.
	Specifically, a 128MB machine may not have enough free space for
	a good buffer cache and the large number of seek/reads caused
	by the smaller hash table can result in seek starvation on /news.

Diablo V1.04

	Fixed a 436 return code into a 431 return code for streaming 
	'try it again' responses.

	Now uses floating point to store and log byte and article totals

	Fixed bug in the article totaling (for logs) - Diablo was reporting
	more received articles then were actually received in the DIABLO
	log line.

	Diablo incoming now logs every 1024 articles rather then every 1000
	for a better looking K/M/G log line.

	Added 'maxcross' and 'maxsize' filters for outgoing feeds.

Diablo V1.03

	Uses mmap() to allocate memory, either via MAP_ANON or MAP_PRIVATE,
	allowing us to easily deallocate it from the page table on fork.
	We use this for stdio buffers as well as for pipe line buffers.

	Added doutq

	More portability fixes:  directory stuff, signal stuff, etc...

Diablo V1.02

	Fixed two file descriptor leaks in the main server.  There are not
	likely to be any more.

	Made portability changes:  uses sigaction() rather then signal(),
	and uses fcntl() rather then flock() (though SUN should get it's
	act together and implement flock().. it's trivial).

Diablo V1.01

	Moved some .SET's from XMakefile to XMakefile.inc

	The sample crontab in adm/ ran the once-every-four-hours
	expire once a minute instead, and I didn't notice!

	dnewslink now reports local/remote latency properly.  Well,
	as well as can be done, anyway, and also reports the size
	of the article in debug mode.

Diablo V1.00

	Initial Release

