DNEWS V4 Conglomerate Manual

Installing and upgrading instructions

Managing Dnews


Chapter 5. The .conf Files

The .conf files are used to configure the system they can be modified directly or on the Windows and Macintosh versions some changes can be made via the gui interface. There are six .conf files

The following files define your news system in approx. order of importance:

File NameDescription
dnews.conf Main configuration file
access.conf Controls nntp access to your server
expire.conf Specifies how long to keep news items
newsfeeds.conf Defines feeds out of your news system up the line of news servers
moderators.conf Defines where to send email to a moderated news group
control.conf Defines how to process control messages for auto creation and deletion of news groups

[return to index]

dnews.conf

SymbolExample Description
active_statictrue Stops new groups from being added by the sucking feed. Use this if your feed site has lots of groups you never want added into your active file.
auth_casetrueTells dnews to be case sensitive when testing usernames and passwords
body_chunk20000Sets the size of chunks to read from disk when a user reads an article the default is 20000
cachedalltrue Pretend all groups are cached use this if you are taking a full IHAVE feed in.
cachedonlytrue Only accepts items into groups which are cached this is used for the hybrid feed where items are sent by a feed but only taken if users are reading that particular group.
cancel_anyonetrue Allows cancel messages to work even if they are not from the original poster
cancel_usersme@hereOnly allow cancel messages from this person
chan_nobufftrue Use to help the Mosaic news reader to work. (it has a bug)
configdnews_top: Directory for configuration files (*.conf)
confirm*.this.place Send email confirmation to postings from these addresses. (set this to match your local email addresses)
db_index2000Sets how many index files to keep in memory each one is 1200 bytes the default is 200. If you have memory to burn then increase this to 3000 for good performance.
db_max200Sets how many files dnews should attempt to keep open to increase performance on Windows NT you can safely set this value to about 200 on most other systems a value of 30 or 40 is about as high as you can go. You can use the command tellnews test_files to see how many free file handles dnews has.
debugchan_read:chan_write: Use to set which debug lines should be printed to the log file. The parameters cannot be separated by spaces. The default setting is

control:db:cancel:file:lib: warn: fatal: post: chan: feed:
hist: nntp: str: err: debug: expire: info: item: group: suck:
cmd: ncmd: init: db: mem: ");

debug_rotate10000 Automatically rotates log files (renames and deletes the oldest one) if more than 10000 lines are written. This is good if you have turned on debugging as you always have the last few thousands lines if a problem develops but you don't run out of disk space due to the huge log file.
db_flush_every1 How many seconds to wait before closing some more files (default 60)
db_flush_n10 How many files to flush when the flush_every timeout occurs increase this to make sure all files are closed decrease it to improve performance. (default 1)
debug_store100Stores the last 100 items as files the items are stored before they are processed this was used for finding items that crashed dnews the last stored item was the problem. It is not an official setting. (i.e. it may be removed)
df/usr/bin/df -PkDefine the df command to use on Unix systems where to find it if necessary and the switches to use if needed.
diskspace100 Total disk space in megabytes a dynamic feed needs 20-200MBs a full feed needs approx. 1000MB
expire_at5 23 3 Run an expire job at 5 past eleven and 5 past 3 in the morning.
expire_bitstrue Allows the expire to run more smoothly by stopping part way thru large files to allow dnews to respond to users requestes. (not generally needed)
expire_chunk100000Sets how many lines from the history file to expire in one go.
expire_delay3The maximum delay before dnews will expire some more items even if it is flat out answering user requests. (not generally needed)
expire_fasttrue Increases priority of expire job to get it done fast in the night.. (not generally needed)
expire_minutestrueUsed to make dnews use the minutes setting in the expire_at rule.
expire_small1000When dnews is using more than 89% of the disk space there is a danger of it deleting toto many items in one go this setting is used to limit how many it deletes before checking to see if the disk is still almost full.
feed_readertrue (default) Sends the 'mode reader' command when connecting to a feeding site this is the default. You will probably never have to change this setting.
fts_dird:\dnews\ftsSets the directory where the fts index files are being stored. (see the section on FTS for more details)
fts_expiretrueAutomatically set if fts_dir is true this tells dnews to write a list of all expired articles so that the fts system can remove them from the fts database.
groups_at20 2Sets when dnews should check for newly created news groups on the site that it is sucking news from. (not needed with an IHAVE feed where the control news items perform this function).
head_cache200How many item headers to keep in memory if you have memory to burn then a setting of 3000 will improve performance of some nntp commands
head_limit2000Limits the size of headers to keep in the cache the default is about 2000
hide_queuetrueStops the list of groups being sucked from being shown in the 'Downloading...' message.
history\dnews\spool Directory for history file this could grow to 40MB for a full feed (80 000 blocks)
history_min5 Pause if less than 'n' megabytes free
item_max100000 Sets the maximum size for any article that DNEWS can deal with the default is 1 000 000. A smaller number will save memory.
life30 Number of days to keep fetching a news group from the upstream server after someone reads an item in it.
list_min300How many groups need to be cached before the update method switches to the technique that works best for large numbers of groups.
log_inlinefalseStops logging of all the 'item ok' messages inside the dnews.log file
log_nntpfalseStops logging all the nntp commands sent to dnews.
logdir\dnews Directory to write log files
logdir_min5 Pause server if less than 5MB is left free on logging disk.
loglvlinfo One of: error warn info debug flush

There are four levels of error reporting "error" includes serious errors "warn" includes possible problems "info" includes general information messages and "debug" which is used for tracing bugs. "flush" is the same as debug but the log file is flushed with every single line written this is useful if the system is crashing but not providing a stack dump.

logusetrue Write a log of every item body read by users in the file used.log
mailsmtp This is only used on VMS systems. The name of the foreign email system on VMS typically in% or smtp% (don't include the %)
mail_fromsystem@your.domain Sets the from address used when dnews sends a report or email confirmation to a user this is somtimes needed by smtp servers which require a valid return address.
managersystem@here.this.place Email address of the news manager
max_out10000Limits the speed that any channel can read news articles in characters per second.
max_speed10000Limits the speed in characters per second that any channel can send news into dnews.
mynamenews.this.place Your full internet name if left out this may be picked up from the system. It's best to put it in by hand to make sure it's right.
nntp_blocktrue Sets all the sockets as blocking we don't know why you would want to do this.
nntp_feedernews.feeder.site The news site where you are going to SUCK a feed from. You need NNTP READER access to this site. If you are not sucking a feed then comment this line out. If you need to specify a PORT other than the default one (119) then add it on the end of the name e.g. feeder.site.edu:2021
nntp_logoffYou are not one of our users Specifies the message to give to users who are not granted connect access via access.conf.
nntp_passmysecretword Use this if the site you are sucking from requires a username and password to access it.
nntp_suck2 3 4second.feeder alt.irc.* myuser mypassword Get irc groups from another news server every 10 minutes the user/passwords are optional. (also see update_groups)
nntp_userjbloggs Use this if the site you are sucking from requires a username and password to access it.
nntp_xpostingtrueFor a posting feed this lets you add a header which will identify any user that attempts to post spam thru your server.
no_newnewstrueDisables the newnews command.
nocontroltruePrevents automatic creation of the 'control' news group.
nodnstrue Stops DNEWS from doing reverse DNS lookups every time a user connects you will have to put ip numbers into access.conf if you use this feature but if your DNS is faulty or slow then this can improve connection times.
nospacechecktrueStops dnews attempting to find out how much disk space is free. If this test is failing your server may pause itself so this setting gets it working again. This is quite safe as the test is only used as a backup check.
nostreamtrueTells dnews to pretend it cannot take a streaming incoming feed.
nwserverhostnameUse on netware servers if NetWin suggests it should be used.
open_blocktrueMakes nntp open calls block the news server
open_retry5How long to wait after failing to open an nntp connection before retrying .
orgNetWin LtdSets the name to be added to local posts which do not have the Organization: header set
org_replacetrueForces local posts to have your organization name even if the news posting software attempted to put some other name in.
out_size4000Sets the size of packets to be sent in one write operation to a socket.
pidenttrueEnables pidentd checking (a way of authenticating users on multi user unix systems)
port119 The TCP port which DNEWS should listen for incoming connections on.
post_logtrueLogs every local message to a post.log file
post_routetrueEnables routing of incoming ihave messagse to outgoing posting feeds.
postonlytrue Stops items from being posted locally until the 'echo' back from the feeder site this gives a very clear confirmation that the message has been passed up stream. You cannot do this if you want to have local groups which are not on the site you are sucking from.
replicatetrueMakes dnews use the same item numbers for each item as are used on the server it's sucking from. The result is a second server that mirrors the first.
reportno Stops the daily email report from being sent to the manager after every expire run.
send_at*5 * Send messages upstream when the time matches this would do it every 10 minutes. e.g. at 1:05 1:15...
send_latertrue Stops DNEWS from attempting to send a post immediately useful if you are using a dial up connection. You can trigger this by hand using 'tellnews requeue'
sendmail/usr/ucb/sendmail Path to sendmail or equivalent software with switches. (only used on unix)
spool\dnews\spool Disk name (or logical) for news articles
spool_min5 Pause if less than 'n' megabytes free
suck_batchn100Maximum number of items to get from one group before going onto the next group.
suck_batchsize1000000Maximum bytes to get from one group before going onto the next group. Use this to stop dnews spending all day on a binary group.
suck_requeuetrueTells dnews to reque a group that is not finished.
suck_streamtrueMakes dnews suck much faster don't set this unless you need to as it hurts the server you are sucking from.
suck_timeout300How many seconds a sucking feed can be idle before dnews should close the connection to try and get it un-stuck.
suck_uucp/uucp_in/*.bagDirectory and file names of incoming uucp files. Every minute dnews will scan this directory and read in any files it finds and then delete them. They should be in standard rnews format.
tcp_nodelaytrueSets the socket to send data immediately this may solve problems with bugy news readers. (e.g. Mosaic news reader)
timeout30 Define how long to wait after a channel becomes idle before disconnecting it the default is 120 minutes.
timezoneNZST Timezone hh:mm or timezone name Appendix B
too_old50 Refuse items that were created more than this many days in the past this prevents old items from being re-distributed when a machine dumps old news items onto the network.
update_at*0 *5 * Check for new items every time the minutes end in a 5 or a zero and at any hour e.g. every 5 minutes..
update_at2 3 410 *Update setting for second third and fourth feeds check less frequently every hour at 10 minutes past.
update_groups1 2 3*binary* 45 1 A way of getting specific groups more frequently only works with the main feed. For example get binary groups once a day at 1:45 this is in addition to the update_at setting not instead of.
uucp_crlftrueWrites uucp files as MSDOS text files instead of proper unix format.
workarea\dnews\spool Directory for work files (active.dat *.feed etc)
workarea_min5 If less than 'n' megabytes are free on this disk then pause the news system (stop accepting new articles)
xover_cache4000How many xover records to cache this is the best way to improve performance if you have spare memory set it as high as 15000.

[return to index]

access.conf

This file controls access to your system you can control access by IP name IP number or even by individual user using password protection.

<host>:<perm>:<user>:<pass>:<groups>

ParameterExample Description
<host>*.my.domain

161.29.4.*

A wildcard name or number this rule is only used if the connection matches this address
<perm>Read Post Ihave logoff Allow any combination of reading posting or ihave feeding from this site.
<user>chrisp Some news readers can send a user name if so then it will be matched here and the password can be checked.
<pass>rumplestiltskin The password in plain text
<groups>* !*.erotic.* Allow access to groups matching this wild card

Notes:

Unlike other news implementations you can permit reading and posting to different combinations of news groups for example you may wish to allow students to read all groups but only post to local news groups.

The last matching line in the file is used. This file can be tricky to setup if it doesn't work as expected check that you have the entries in the right order so that the last matching ip name does what you want.

[return to index]

expire.conf

This file can be tricky to use remember order is significant!!

The expiration file is very different from INN and CNEWS implementations Rather than a simple set of rules the DNEWS expire rules match both news group names and current disk usage so as use increases the rules can become more strict.

In addition rules can specify minimum or maximum numbers of items this allows a single rule to apply to groups which get 20 items a day or 20 items a month.

Each line of the file must have the following format:

UPTO x GROUPS wild KEEP n DEFAULT n PURGE n MAXITEMS n MINITEMS n
SymbolExample Description
upto20% This rule applies if less than 20% of the disk space is full.
groupsalt.* comp.binaries.* This rule applies if the group matches this list
keep2 Keep items for this many days even if they have already expired. (From the expire date in the article header)
default10 Keep items this long and then delete unless the expiration date says to keep it longer.
purge35 Delete after this many days even if the expire header says to keep it longer. This should be set long enough to keep FAQ's and no longer to prevent abuse.
maxitems1000 Delete old items if the group has more than this many items in it. Good for controlling high volume groups.
minitems10 Don't apply any other rules if the group has less than this many items in it this is good for making 'low volume' groups useable.

Example:

# Default rule
upto 10000% groups * keep 0 default 1 purge 10 maxitems 0 minitems 0
# Start to panic a little
upto 90% groups * keep 0 default 2 purge 33 maxitems 100 minitems  0
# Normal operation
put your meaningfull settings here.
upto 60% groups * keep 2 default 5 purge 33 maxitems 1000 minitems  10
upto 60% groups comp.* keep 2 default 10 purge 33 maxitems 2000 minitems  10
# Lots of space
just stop things getting silly (e.g. 10000 items in...)
upto 30% groups * keep 2 default 17 purge 33 maxitems 3000 minitems  100
upto 20% groups * maxitems 2000
# Keep any local groups forever (this line must be last to work)
upto 1000% groups local.* keep 99999 minitems 10000

If disk use is 58% then the rules that will be applied are the upto 60% rules once the useage goes past the upto value of a rule the rule is no longer matched on. DNEWS will use the last rule it finds that matches searching from the top down. The upto 60% rules comprise of one rule for all groups and one rule for the comp.* hierachy the specific rule must be after the general rule or it wil never be matched. DNEWS will apply the rule like this if the item has an expire header DNEWS will apply the keep rule and keep the item for 2 days even if the expire header says not to. The default rule is the rule that gets applied to all of the articles DNEWS will expire an item that it has had for more than this many days unless the expire header says not to for the comp.* groups this is 10 days for all of the other groups 5 days is applied. The purge setting removes anything that has been on the server for this many days even if the expire header says not to this setting is used for removing periodic posts such as FAQ's. The maxitem and minitem settings overide the other settings and will keep or remove items even if the other rules say not to. There are cases where the maximum and minimum counts may seem to not be applied this is usually due to crossposts.

[return to index]

newsfeeds.conf

This file controls feeds out of DNEWS to other sites it also controls the feed 'into' DNEWS via the same mechanism.

SymbolExample Description
siteme

$NNTP_FEEDER

my.leaf.site

The name of the site to feed usually it's full internet name. The 'me' feed must be the first one in the file.
builtin(flag no value) Specifies the builtin feeding mechanism should be used. This is only intended for sending local postings up stream it is not fast enough to send a full feed on to another site.
posting(flag no value) Specifies that the nntp 'post' command should be used to send items upstream. This gives your feeder site the job of deciding how to email postings to moderated groups.
excludefeeder.real.name feeder.third.name Use this if your feeder site is 'name stamping' articles with a name other than it's ip name. This MUST be right or DNEWS will attempt to echo everything it gets back to the feeder site. (check the path header)
smtp(flag no value) Specifies this is not really a feed but on a vax it is actually the email queue used to send articles to moderated groups this is only needed on VMS and then only if you are not using the 'POSTING' mechanism to feed upstream.
maxcross50 Throw away items which are crossposted to more than '50' news groups. This is good for preventing 'spamming' which is where people post adverts and money making schemes to inappropriate news groups.
maxsize100000 Limit items to 100K the default is 1MB if you want to make it larger than 1MB then you must also increase ITEM_MAX in DNEWS.CONF. Personally I think 100K is too big ftp sites and web should be used for larger items. Dnews will still download the items it just doesn't store them if they are larger than this setting.

Setting your EXCLUDE parameter correctly:

In newsfeeds.conf you must set your exclude parameter to stop DNEWS from sending incoming messages back to your feed site. Lets assume your feed site is called fastfeed.world.net. You need to find out what it stamps onto news articles to do this read a news article (and show all headers so you can see the PATH header)

e.g.

Newsgroups: alt.winsock.trumpet
Subject: Sportster Si does not seem to work with TRUMPET
Date: Thu
26 Oct 1995 22:44:08
Organization: Personal
Lines: 8
Message-ID: <moleski.2.0016BD02@sasknet.sk.ca>
NNTP-Posting-Host: eagle28.sasknet.sk.ca
X-Newsreader: Trumpet for Windows [Version 1.0 Rev A]
Path: fast1.world.net!world.net!news.icix.net!esol.intermedia.com 

Look thru the path left to right and find the first or most generic name for your feed site in this case 'world.net' looks like the best name to choose.

So here is the entry you should put in newsfeeds.conf you may need the builtin and posting settings as well but that is up to you.

site fastfeed.world.net groups * exclude world.net builtin

You may need to delete your .feed and .sending files from the spool directory to clear out the backlog of articles queued.

[return to index]

moderators.conf

You won't need to touch this file unless you want to add a local group and make it moderated. In that case add your groups at the top of the file as the first match is used.

Format:

newsgroup:email-address

If the email address contains a "%s" then that is replaced by the news group name with the dots changed into dashes.

Example:

local.gossip.moderated:chrisp@marc.cri.nz
gnu.*:%s@tut.cis.ohio-state.edu
*:%s@uunet.uu.net
[return to index]

control.conf

This file controls the action taken when control messages are received. This is the secret to the administration of news. Control messages are just like normal Usenet news articles but they are read by your news server and make it automatically do things like adding and deleting news groups. This may seem invasive but with dozens of news groups being created and deleted every day you really don't want to be doing this yourself.

See the section on Adding PGP authentication to control.conf if you are concerned about secuirty issues involved with control messages. However there is not too much need for concern DNEWS does not actually remove anything if a news group is deleted so it can always be recreated if necessary.

Cancel messages are not processed by this file for performance reasons they are dealt with differently.

The following control messages are checked by this file:

However only the following three are implemented.

Blanks are not allowed e.g. :: would be misinterpreted.

For each type of control message you can specify based on who sent it whether or not to do what the message says just log it or email the message to the news administrator. The log file that DNEWS creates can be used to action the control messages at a later date as it contains valid commands.

All control messages are logged to 'control_(message).(action)'

Format:

message-type:email-address:action
message-type    rmgroup
newgroup
version
email-address   A wildcard email address
e.g. news@trusted.site
action          "drop"
"log"
"mail"
"doit"
"doit
mail"
"doit
pgp"

Example:

all:*:*:mail
ihave:*:*:drop
sendme:*:*:drop
sendsys:*:*:drop
senduuname:*:*:drop
newgroup:tale@*.uu.net:comp.* misc.* news.* rec.* sci.* soc.* talk.*:doit
rmgroup:tale@*.uu.net:comp.* misc.* news.* rec.* sci.* soc.* talk.*:doit

[return to index]

Chapter 7. Adding More Feeds

There are a number of combinations and permutations involved when it comes to adding additoional feeds to an existing feed

Add a second suck when the existing feed is a suck. Add an Ihave feed to an existing suck. Add a suck feed to and existing Ihave feed. Add another Ihave feed to an existeing Ihave feed.

7. Complex Feeds

Dnews can be used with different kinds of feeds such as a feed via satellite or from a bbs or some other such thing that's not a regular news feed these are generally in a UUCP format.

Xmit - a main page for the appendices ???

What is UUCP ??? - add the relevant section here
[return to index]

Gatewaying Mailing Lists

CAUTION gatewaying mailing lists to public news groups is a little risky don't do it lightly.

Two gateways are needed to make a news group and a mailing list 'identical' you must setup both so that a news post gets sent to the mailing list and so a mailing post gets posted to the news group.

Setting up a gateway from a "NEWS group" to a "MAILING LIST"

In newsfeeds.conf add a feed for the group you wish to gateway the 'site' name can be anything meaningful to you.

site juggling groups rec.juggling< (This will make dnews create juggling.feed in the workarea directory)

Now to actually send the items setup an xmit process for each mailing list. To do this edit addmail.cmd (which will be in /dnews with luck)

notepad addmail.cmd
Change the -name to be the name in newsfeeds.conf
Change MAIL.HERE.COM to point to your mail server
Change JUGLIST@HERE.COM to point to the actual mailing list
addsvc -del dxmail1
addsvc -add dxmail1 xmit.exe

"dxmail1 MAIL.HERE.COM -at *5* -mail (should all be on the same line)JUGLIST@HERE.COM -name juggling"
addmail.cmd (run the script to add the service)
Restart things:
tellnews reload
Start the xmit service in Control Panels (for nt)
Test the mailing list if a loop occurs disable it quickly and figure out why.

Setting up a gateway from a MAILING LIST to a NEWS GROUP (Windows NT only)

First setup DROBOT to run from ntmail in control panels run the NTMail admin tool. Click on 'executables' and enter a name for this gateway e.g.

juglist-robot

(You will then need to add this user to your juglist mailing list e.g. juglist-robot@here.com so that this robot will get all messages that are posted to the mailing list)

In the command line box type in:


drobot -dir c:\ntmail\news -groups rec.juggling
Now setup the service to post these items.
addsvc -del dposter
addsvc -add dposter dposter.exe "dposter -dir c:\ntmail\news -to news.here.com"

Now start the services (from control panel) and then test posting a message and mailing a message to the mailing list and see if the newsgroup and mailing list are kept inline.
[return to index]

Setting up a UUCP feed into DNEWS

In dnews.conf add

suck_uucp c:\uucp\in\*.bag

The BAG/UUCP file format is:

#! rnews nnnn

...(article exactly nnnn bytes counting each end of line as one byte)

#! rnews nnnn

...(next article)...

Setting up a UUCP feed out of DNEWS

In newsfeeds.conf add a feed e.g.

site site1.name.uucp

type uucp

uucp_dir c:\uucp\out

groups *

This will create files called news000n.bag in the directory c:\uucp\out It's your job to uucp copy these to the destination sites and delete them. If you want you can compress them first. :-)

The format of the file is:

#! rnews nnnn

...(article exactly nnnn bytes counting each end of line as one byte)

#! rnews nnnn

...(next article)...

[return to index]

The XMIT Process

(XMIT (Setting up full feeds)

This is necessary when you want to send a full feed onto another site. If you are a leaf site then you DONT NEED TO DO THIS !!!!!! First add entries to your NEWSFEEDS.CONF file this creates the file site_name.feed then you need to setup a timed job to send those items to the other sites using NNTP IHAVE (or streaming extensions)

You should set all groups as cached in dnews.conf add a line:

cachedall true

XMIT (On unix)

Add lines to your rc.local procedure e.g.

cd /usr/local/dnews

./xmit send.to.site.name -at "*2" "*"

./xmit send.other.site.name -at "*7" "*"

XMIT (On Windows NT)

On windows NT you need to setup each xmit job as a service to do this use the program ADDSVC.EXE which came with the dnews distribution to add a service for each outgoing feed e.g. (see setupsvc.cmd)

addsvc -add dxmit1 xmit.exe "dxmit1 site.name.edu -at *5 *"

addsvc -add dxmit2 xmit.exe "dxmit2 site2.name.edu -at *5 *"

Then from CONTROL PANEL SERVICES and start the feeds.

XMIT (On VMS)

See RUNXMIT.COM
[return to index]


Chapter 4. Tellnews - controlling the dnews server

Tellnews commands can be used in all versions of dnews they are just typed in on the command line with the Windows NT and Macintosh versions they are also available throug the Dnews Manager interface.


Command: addlost

Syntax: tellnews addlost

Description: A very useful command used when the history file and the database get out of synch re-enters into the history file items that are not indexed. Also corrects the range of items for each group (as stored in the active.dat file)

This command runs as a background task like an expire it will take several hours per gig of news. So a large system may take 10-24 hours to complete.

This command is obsolete in version 4


Command: cached

Syntax: tellnews cached group_wildcard n_days

Description: Causes the group or groups listed to be cached on the server for the number of days specified.

Example: tellnews cached rec.bicycles.* 3

This will cause dnews to update the rec.bicycles hierarchy every day for 3 days even if no one reads any of those groups.


Command: check_index

Syntax: tellnews check_index

Description: Takes a few seconds checks the index and corrects the group ranges to match the index.

Version 4 command.


Command: checkgroups

Syntax: tellnews checkgroups check_news_item.txt

Description: Processes the given file as if it had been recieved as a checkgroups message. A checkgroups message is a list of 'real' news groups in a given news hierachy for example all the rec.* news groups. Any news group that is not in the check groups messages is deleted any news group which is missing is added.

Example: tellnews checkgroups message.txt


Command: clear_cache

Syntax: tellnews clear_cache

Description: Clears some of the internal caches used for reading item files.


Command: crash_now

Syntax: tellnews crash_now

Description: very dodgy command used to simulate the behaviour of other companies news servers.


Command: debug

Syntax: tellnews debug [chan_read:chan_write:… ]

Description: Used with loglvl debug or flush to control which debugging messages are sent to the log file. The list of options must not contain spaces. The default list is:

control:db:cancel:file:lib: warn: fatal: post: chan: feed: hist: nntp
str: err: debug: expire: info: item: group: suck: cmd: ncmd: init: db: mem:

Example: tellnews debug chan_read:chan_write:


Command: disconnect

Syntax: tellnews disconnect n (where n is the number of the channel to disconnect)

Description: Used by the dnews admin to disconnect a single channel select the number 'n' by doing a tellnews status co


Command: diskuse

Syntax: tellnews diskuse comp.os.*

Description: Gives the disk usage of all the groups in that directory tree. This total counts each cross post so the total can be larger than your entire system. Where "comp.os.*" is any group/subgroup that you wish to specify.


Command: dns_clear

Syntax: tellnews dns_clear

Description: Used by the dnews admin to clear DNEWS's cache of ip numbers and names. Very rarely necessary. If you change the names or numbers of local machines then this command should be used.


Command: dns_show

Syntax: tellnews dns_show

Description: Used by the dnews admin to inspect DNEWS's internal cache of ip numbers and names. Very rarely useful for normal operation unless you are having trouble granting access to a particular ip name and you need to see what name DNEWS is finding for that computer in the DNS.


Command: doupgrade

Syntax: tellnews doupgrade

Description: NEVER use this command. It was for converting the DNEWS database from version 1.0 to 2.0. It will also be used to force an upgrade from version 2.0 to version 3.0 however since this is automatic there is never any need to use this command.


Command: exit

Syntax: tellnews exit

Description: Shuts down the DNEWS server cleanly.


Command: expire

Syntax: tellnews expire

Description: Used by the dnews admin to force an expire run during the day this is occasionally necessary if something has prevented normal expire runs to work and after fixing the problem you wish to start an expire run immediately to free up disk space. The expire run will begin as soon as the current entry in the sucking queue has completed.


Command: fileonly

Syntax: tellnews fileonly

Description: Used by Netwin support staff to test dnews performance. NEVER use this command.


Command: findlost

Syntax: tellnews findlost

Description: corrects the range of items in active.dat based on existing .idx files this is similar to 'addlost' but it is much faster as it doesn't also add the items to the history file so use it when active.dat is messed up but the items are in the history file. (added at ver 3)


Command: fixnames

Syntax: tellnews fixnames

Description: Used by dnews admin to fix the active.names file. This command will check the file is internally consistent and it will remove any duplicate entries. This command should be used every few months to keep the system clean.


Command: fixrange

Syntax: tellnews fixrange

Description: Tells dnews to forget how many items it thinks each group contains and actually check the next time a user enters the group. This is somtimes useful if winvn is crashing because dnews has lost track of how many items exist in a particular group.


Command: fixtimes

Syntax: tellnews fixtimes days_old

Description: Tells dnews to re-write the active.times file and pretend that all groups were created 'days_old' ago. This command should be used once every 6 months to clean out old groups from the active.times file.


Command: getgroups

Syntax: tellnews getgroups [n]

Description: Used with a sucking feed to get the list of groups from the provider if there are more than 300 groups cached dnews will do this automatically every update_at period if there are less than 300 groups cached it will do this command automatically once per day. It can also be used to get the groups lists from additional feeds by specifying 2 3 or 4 as n. This has to be used to add an additional feed to dnews.

When you first start up dnews it will also try and get the full list of groups but if it fails on the first attempt (due to some temporary problem e.g. you forgot to plug in the ethernet cable) then use this command to force it to fetch the list again.

Example: tellnews getgroups 2


Command: getnames

Syntax: tellnews getnames

Description: Gets the list of news group names from the news server you are sucking from and stores them in the active.names file.

Very few news readers display these full names so it is not critical to get this.

Example: tellnews getnames


Command: getold

Syntax: tellnews getold <group wildcard> n

Description: DNEWS keeps track of the latest item number for every group on the site it is sucking from. This command lets you adjust this counter downward by 'n' for every matching group this can be useful as a way to 'refetch' items that were somehow missed previously.

This command will not result in duplicate articles because of the history file dnews will not refetch an article if it's message id is stored in the history file.

Example: tellnews getold rec.bicycles.tech 100


Command: group_age

Syntax: tellnews group_age

Description: Age all groups this command emulates the passing of day. Each cached group has a counter of how many days it should remain cached for and this command will reduce every counter by one and will then 'un-cache' any groups who's counter has reached 0.

Example: tellnews group_age


Command: group_merge

Syntax: tellnews group_merge

Description: Merge a file called active.add (in the workarea directory) with the existing active.dat file. This is the correct way to update your active list to include groups from a new site that is now sending you items via IHAVE. Don't edit the active.dat file by hand :-)


Command: group_nodup

Syntax: tellnews group_nodup

Description: Checks the active file and removes any duplicates this command should never be necessary it was added during an early version of dnews which had a bug which could allow duplicate groups to get into the active file.


Command: hist_test

Syntax: tellnews hist_test

Description: Shows in the log file every time an actual history file read is necessary this is was used to confirm that history lookups were as rare as possible while accepting an ihave feed.


Command: key

Syntax: tellnews key

Description: Enter a license key to the server the key is letter-number-letter-… and all of the letters are lower case. There are a few different responses this command may give that indicate whether the key has been entered succesfully.

Example: tellnews key a6j5a0a2a4p6a9


Command: killgroup

Syntax: tellnews killgroup <group wildcard>

Description: Removes the specified group(s) from the active file you must shut down and restart dnews after using this command. This command does not attempt to delete the items related to the group this command is intended for use only when you know best.

To cleanly remove a news group use the rmgroup command (which is reversable) and then wait several days or weeks for the expire process to delete the items in the now deleted group.

Example: tellnews killgroup netwin.chat.group


Command: killitem

Syntax: tellnews killitem <group name item number | "message id">

Description: Delete an item in a group the item is defined either by it's message id or by a combination of it's number and the group it is in this information can be found by using a newsreader such as WinVN that shows the article numbers. Usually used for removing inappropriate postings from private local groups.

Example: tellnews killitem "<540kfg$75p@hplms2.hpl.hp.com>"

Example: tellnews killtitem rec.bicycles.tech 57536


Command: log

Syntax: tellnews log

Description: Display the last 50 lines of the dnews.log file.


Command: Loglvl

Syntax: tellnews loglvl <debug | flush | info | warn | error>

Description: Set the level of messages to be recorded in the dnews.log file can be used in conjunction with the tellnews debug command. This command is very useful for enabling a higher level of debugging for a brief period while testing some problem system.

Example: tellnews loglvl debug

Example 2: Testing a problem with report messages not arriving:

        tellnews update_clear
tellnews loglvl debug
tellnews debug chan_read:chan_write:suck:
tellnews report
        (wait 20 seconds)
        tellnews loglvl info
        (now examine the log file in detail)

Command: matchfeed

Syntax: tellnews matchfeed

Description: Marks all groups that dont match the groups entry under site me in the newsfeeds.conf file as deleted. This is typically used to change the me feed to allow groups that were previously blocked (in conjunction with tellnews undelete) or to set newly blocked groups to deleted.

Example:

        notepad newsfeeds.conf
tellnews reload
tellnews undelete *
tellnews matchfeed


Command: newgroup

Syntax: tellnews newgroup <groupname> [y |m][creator][description]

Description: Add a new group to the group list this can be a local group or a new group from another feed site or a group from the upstream feed. The optional y or m field is to set the group as normal (y) or moderated (m) the optional creator field is a one word name of the groups creator the optional description field is a few words describing the group.

Example: tellnews newgroup local.bicycle.rides y eddym Info about bike rides in our vicinity


Command: purgelost

Syntax: tellnews purgelost

Description: Removes all of those articles in Dnews's database that don't have an entry in the history file very useful if the system has become very full due to some condition that caused it to loose a number of entries from the history file. Generally speaking addlost is a better choice to fix a mixed up dnews database.

This command is obsolete in version 4


Command: purgegroups

Syntax: tellnews purgegroups

Description: Actually remove and forget about groups that have been marked for delete. You must stop and start dnews after using this command. If you use this on a sucking feed and the deleted groups exist on the site you are sucking from then the groups will re-appear.


Command: purgelinks

Syntax: tellnews purgelinks

Description:Similar to the 'addlost' command this command checks the integrity of the dnews database for each cross posted article it checks that the actual article still exists if it does not then the cross post is deleted. This command should only be necessary of you have corrupted the database by deleting item files by hand.


Command: rebuild_hist

Syntax: tellnews rebuild_hist

Description: Add any items to the history file that arenot already in it. (rebuild_hist should not generally be used alone as the history and index files are co-related they should be rebuilt together with rebuild_index see below)

Version 4 command


Command: rebuild_index

Syntax: tellnews rebuild_index

Description: Rebuilds index and the history database from scratch the items will disappear then reappear during the rebuild. (rebuild_hist should not generally be used alone as the history and index files are co-related they should be rebuilt together with rebuild_index)

Version 4 command


Command: refeed

Syntax: tellnews refeed feed.name

Description: This command will re-send all current articles to a particular news feed this command is fairly cpu intensive as the entire history file (maybe 80MB) must be parsed. Also it does not apply all the rules associated with the feed you have specified it only checks that the group name matches. This command us particularly useful when setting up the FTS (Full Text Searching) system on a running server. It allows you to 're-index' all existing items.


Command: reindex

Syntax: tellnews reindex

Description: In theory this command is like 'addlost' but instead of working off the .idx files it reads the .itm files and recreates the .idx files. In practice it does not yet work correctly except to fix up problems caused by a bug in 3.0a. In future it will be modified to rebuild the .idx files under any conditions.


Command: reject

Syntax: tellnews reject "<message@id.host>"

Description: Tells dnews to immediatly reject the specified item if it is offered it by an ihave feed. This command is very rarely useful if a particular item is blocking an incoming feed for some unexplained reason.


Command: reload

Syntax: tellnews reload

Description: Reloads the .conf files necessary to make any changes to the files take effect if not done the changes will not take effect until you restart dnews.


Command: reload_fts

Syntax: tellnews reload_fts

Description: Tells dnews that the fts index has changed and should be reloaded from disk. This command is not usually needed the index is normally re-read fairly often automatically.


Command: report

Syntax: tellnews report

Description: Forces dnews to generate the expire report and mail it to the manager address set in dnews conf. Very useful if you are not getting these reports it allows you to check the log file and see why the report isn't successfully sent.


Command: reque

Syntax: tellnews reque

Description: Puts all outgoing feeds into the sucking queue and attempts to send them. Again this is useful if a builtin outgoint feed or smtp feed is not working it allows you to trigger a retry and watch to see why it fails.


Command: resetall

Syntax: tellnews resetall

Description: Use with care this resets all of the article counts in the active.dat file the main use for this commnad is to clear an active file that has been ftp'd from some place other than your provider and the numbers are meaningless. NEVER NEVER NEVER use this command on a working dnews server it is only intended for use during the initial installation to clear out an active file from a unix system. To add groups from someone elses active file use the group_merge command described above.


Command: resort_index

Syntax: tellnews resort_index

Description: Sorts the index so any group can be read with a single disk scan this will also remove deleted entries from the index.

Version 4 command


Command: rmgroup

Syntax: tellnews rmgroup <group name>

Example: tellnews rmrgroup local.secret*

Description: Flags the specified group(s) as deleted. This is the clean way to delete a group and if you make a mistake you can undelete the groups.


Command: rotate

Syntax: tellnews rotate

Description: Manual command to rotate the log files and remove the oldest one there is an automatic setting for doing this in the dnews.conf file (debug_rotate n) Moves dnews.log to dnews2.log dnews2.log to dnews3.log etc.


Command: setused

Syntax: tellnews setused <n Mb>

Description: Set the diskspace used to n Mb n has to be manually calculated and should be the actual amount of news on the system. Used to correct problems caused if the actual disk usage gets out of synch with what dnews thinks it has used. Related to the diskspace setting in dnews.conf and displayed in the output of tellnews.status. For example if the expire process is not removing much from the system because dnews thinks it has only used 10% of the available diskspace but the feed is stopped because there is no space left setused can be used to tell dnews that it has used all of the space and the expire process will then take care of removing articles until the system is functioning normally again.

Example: tellnews setused 570

Lets say you have a 2 Gig disk and your diskspace setting is 1500. 500mb is used by other software but the disk is now full. A tellnews status shows:

750mb used (50%)

You would then do this:

tellnews setused 1500

Wait two minutes and do a tellnews status to see if dnews now as the correct disk use showing.


Command: showcached

Syntax: tellnews showcached

Description: Displays a list of the groups that are cached on the system and the number of days that they will continue to be updated for. At the end of the update time no more new articles will be fetched for the group and the expire process will gradually remove the items from the group. The cached life of the group is reset to the value specified for the life parameter in dnews.conf each time an article from the group is read.

Output:


Command: showmem

Syntax: tellnews showmem

Description: Shows how much memory dnews is using in great detail note: the value under strdup is not accureate. The output of thiscommand is useful to NetWin support staff if you are having a problem with memory use. (see also the related command test_mem)


Command: showqueue

Syntax: tellnews showqueue

Description: Display a list of groups waiting in the queue to be updated.

Output:


Command: speed

Syntax: tellnews speed

Description: Shows how fast articles are arriving. Use in conjunction with tellnews speed_reset for performance testing.


Command: speed_reset

Syntax: tellnews speed_reset

Description: Resets the counters for testing how many articles are arriving per second.


Command: speed_test

Syntax: tellnews speed_test

Description: This command acts as a toggle it is used to test how fast articles would arrive if dnews didn't have to store them on disk the items are thrown away use at your own risk Normally used in conjunction with 'tellnews speed' to determine if dnews is slowing down the incoming feed or if it is limited by some external bottleneck (e.g. the network or the sending machine)


Command: spool_convert

Syntax: tellnews spool_convert /var/spool/news

Description: use this command to convert an existing INN or CNEWS spool area as each file is processed it is deleted you should comment out your feeds before doing this or you may end up re-feeding all this old stuff to your outgoing feeds. (added at version 3)


Command: stats

Syntax: tellnews stats

Description: Returns a listing of statistics showing which nntp commands have been used and how long dnews has taken to process each command.

Explain stats output here with example:


Command: stats_in

Syntax: tellnews stats_in

Description: Returns a report about what dnews has sucked/accepted since the last expire.

Explanation and example output here


Command: stats_groups

Syntax: tellnews stats_groups

Description: Shows the top 40 groups by the number of times they have been read by users and by the volume of articles recieved into that group. This allows you to spot groups which are gobbling up disk space and bandwidth but are not being read by significant numbers of users.


Command: status

Syntax: tellnews status

Description: Returns a report about the status of the system explanation and example output here.

200 161.29.2.10 DNEWS Version 3.0f
posting OK
#This line shows the name and version number of the server
it
#also tells you if posting from your ipaddress is possible.
Feed[0]: me *
!to.*
!*.binaries.*
Feed[1]: 161.29.2.5 Posting Builtin Posts=0 Rej=0 Req=0 Dup=0 file=100k Del=10MB *
# This is a list of the feeds in newsfeeds.conf
the first feed must be called 'me'
# it specifies what groups to accept locally
the other feeds specify where to
# send items and what items to send
(*=send all groups)  The word 'posting' means
# feed[1] is to send items upstream using the posting mechanism
and it is a builtin
# feed (e.g. doesn't need an extermal xmit process to send items)
# The number of Posts
Rejects
Reque'd and Duplicate messages sent out that
# feed since startup are shown.  Also if there is a queue of waiting articles
# then the size of the queue'd file is also shown.
# The 'del' figure is how much queued articles have been deleted because they
# couldn't be sent (This is automatic if the queue grows past 10MB)
Chan[0:5] B o=130 in=1/s out=6/s 161.29.2.10 test.netwin.co.nz C_CMD
Chan[1:5]   o=0 in=13000/s out=600/s 161.29.2.3 test.netwin.co.nz C_IHAVE
# This is the list of currently connected sessions
the ip number and name are shown
# but in this case reverse translation is disabled so it shows the number twice.
# B = Channel busy
as in it is still processing a command and producing output.
# o=130       Output buffer has 130 bytes.
# in=13000/s  Input speed has been 13K /s for the last 30 seconds
# out=600/s   Input speed has been 600 bytes/s for the last 30 seconds
# C_CMD = The channel is waiting for a normal NNTP command
# C_IHAVE = The channel is part way thru an IHAVE transaction.
Expire started at Fri Feb 16 13:31:05 1996 Done
start 1% end 1%
Processed 920 removed 0 deleting 200/-1/150 speed 170.429
Expire finished at: Fri Feb 16 13:31:12 1996
Elapsed 0 hours
0 minutes
# Shows the start and finish time and %'s of the last expire run.
# Processed = number of history lines in your history file.
# removed = number of history lines removed
# deleting shows the number of items being deleted
total and actual
# (of 200 items deleted 50 were cross posts so dont count as real deletes)
# The -'ve 1 is a progress counter which ticks over when the expire is running.
# Speed = history articles processed/second
Contents of DNEWS group fetching queue
2 entries:
Group rec.humor.funny    fetching
got 20 items
Group comp.os.vms    pending (low priority)
Group comp.lang.java    pending (low priority)
End of queue.
# This is the list of groups dnews is currently trying to fetch.
Reverse lookup took 80 seconds
disable lookups if possible (see nodns)
# This error means your name server is faulty (or not contactable)
you
# should disable name translation
set NODNS TRUE in dnews.conf
History: disk 76k memory 7k/80k
Str 2k/5k/500k Malloc 1097k/1097k/2128k
# Current history file is 76K (40mb would be more normal) and the in memory
# amount for this file is 7k (5mb would be more normal)
# The Str and Malloc's are for debugging
they show the total memory allocated and
# also what is being used at present.
Used 1MB=1%
Free work 34Mb
hist 34Mb
spool 34Mb
log 34Mb
# Disk space used as a percent and how much is actually free.
Groups Cached 3 of 4624
Uptime 0 day(s) 0 hour(s)
# Shows how many groups are cached
and how long the system has been running.

Command: test_files

Syntax: tellnews test_files

Description: Used on unix systems for checking on memory problems checks to see how file handles dnews can allocate.


Command: test_mem

Syntax: tellnews test_mem

Description: Used for checking on memory problems checks to see how much memory dnews can allocate. If it cannot allocate at least 20 or 30 MB then your page file is probably not big enough.

You MUST restart dnews after using this command to allow the memory to be freed.


Command: test_sleep

Syntax: tellnews test_sleep n

Description: Locks up dnews for n seconds (only works on Windows NT). This is used to test behaviour of dnews under certain load conditions.


Command: test_sockets

Syntax: tellnews test_sockets

Description: Tests how many sockets dnews can open.


Command: trimgroups

Syntax: tellnews trimgroups n

Description: Similar to the getgroups command but instead of adding missing groups this command removes groups that no longer exist on the site you are sucking news from. Groups will not be deleted if they don't match the wild card in dnews.conf for that feed in this way local groups will probably not be deleted :-)


Command: uncached

Syntax: tellnews uncached <group wild card>

Description: Sets the specified groups as not cached.


Command: undelete

Syntax: tellnews undelete <group wildcard>

Description: Mark a previously deleted group as undeleted


Command: update update2 update3 update4

Syntax: tellnews update[n]

Description: Cause an update from the specified feed to be queued only used for sucking feeds. New items in any cached groups will be fetched.


Command: update_clear

Syntax: tellnews update_clear

Description: Clear the update queue


Command: updateone

Syntax: tellnews updateone <group wildcard> [n]

Description: Add the specified group or groups from the feed [n] in the update queue n is an optional value from 1 to 4 it will default to the main feed if there is no n specified.

Example: tellnews updateone rec.bicycles.* 2 get the rec.bicycles hierarchy from the second feed.

[return to index]


Chapter 5. The .conf Files

The .conf files are used to configure the system they can be modified directly or on the Windows and Macintosh versions some changes can be made via the gui interface. There are six .conf files

The following files define your news system in approx. order of importance:

File NameDescription
dnews.conf Main configuration file
access.conf Controls nntp access to your server
expire.conf Specifies how long to keep news items
newsfeeds.conf Defines feeds out of your news system up the line of news servers
moderators.conf Defines where to send email to a moderated news group
control.conf Defines how to process control messages for auto creation and deletion of news groups

[return to index]

dnews.conf

SymbolExample Description
active_statictrue Stops new groups from being added by the sucking feed. Use this if your feed site has lots of groups you never want added into your active file.
auth_casetrueTells dnews to be case sensitive when testing usernames and passwords
body_chunk20000Sets the size of chunks to read from disk when a user reads an article the default is 20000
bucket_size10000000Sets the size of each bucket optimally you want more than 10 buckets and less than 3000. 10MB is the default for spool areas over 100MB in size. Ver 4
cachedalltrue Pretend all groups are cached use this if you are taking a full IHAVE feed in.
cachedonlytrue Only accepts items into groups which are cached this is used for the hybrid feed where items are sent by a feed but only taken if users are reading that particular group.
cancel_anyonetrue Allows cancel messages to work even if they are not from the original poster
cancel_usersme@hereOnly allow cancel messages from this person
chan_nobufftrue Use to help the Mosaic news reader to work. (it has a bug)
configdnews_top: Directory for configuration files (*.conf)
confirm*.this.place Send email confirmation to postings from these addresses. (set this to match your local email addresses)
db_index2000Sets how many index files to keep in memory each one is 1200 bytes the default is 200. If you have memory to burn then increase this to 3000 for good performance.
db_max200Sets how many files dnews should attempt to keep open to increase performance on Windows NT you can safely set this value to about 200 on most other systems a value of 30 or 40 is about as high as you can go. You can use the command tellnews test_files to see how many free file handles dnews has.
debugchan_read:chan_write: Use to set which debug lines should be printed to the log file. The parameters cannot be separated by spaces. The default setting is

control:db:cancel:file:lib: warn: fatal: post: chan: feed:
hist: nntp: str: err: debug: expire: info: item: group: suck:
cmd: ncmd: init: db: mem: ");

debug_rotate10000 Automatically rotates log files (renames and deletes the oldest one) if more than 10000 lines are written. This is good if you have turned on debugging as you always have the last few thousands lines if a problem develops but you don't run out of disk space due to the huge log file.
db_flush_every1 How many seconds to wait before closing some more files (default 60)
db_flush_n10 How many files to flush when the flush_every timeout occurs increase this to make sure all files are closed decrease it to improve performance. (default 1)
debug_rotate_in1000Number of lines before .in file should rotate. Ver 4.
debug_rotate_out1000Number of lines before .out file should rotate. Ver 4.
debug_store100Stores the last 100 items as files the items are stored before they are processed this was used for finding items that crashed dnews the last stored item was the problem. It is not an official setting. (i.e. it may be removed)
df/usr/bin/df -PkDefine the df command to use on Unix systems where to find it if necessary and the switches to use if needed.
diskspace

diskspace2

diskspace3

..........

diskspace8

diskspace 100

diskspace2 50

diskspace3 60

diskspace4 1000

Define the size of each disk area. diskspace2 ... diskspace8 are Version 4 commands.

Total disk space in megabytes a dynamic feed needs 20-200MBs a full feed needs approx. 1000MB

See Spool below.

expire_anytrueIf a link of a cross post should be deleted then all will be deleted this fixes the problem of people cross posting binary messages to non binary groups and thus filling up your disk.
expire_at5 23 3 Run an expire job at 5 past eleven and 5 past 3 in the morning.
expire_bitstrue Allows the expire to run more smoothly by stopping part way thru large files to allow dnews to respond to users requestes. (not generally needed)
expire_chunk100000Sets how many lines from the history file to expire in one go.
expire_delay3The maximum delay before dnews will expire some more items even if it is flat out answering user requests. (not generally needed)
expire_fasttrue Increases priority of expire job to get it done fast in the night.. (not generally needed)
expire_min10This sets the maximum wasted space in the bucket files the value is a percentage the default is 10%
expire_minutestrueUsed to make dnews use the minutes setting in the expire_at rule.
expire_small1000When dnews is using more than 89% of the disk space there is a danger of it deleting toto many items in one go this setting is used to limit how many it deletes before checking to see if the disk is still almost full.
feed_readertrue (default) Sends the 'mode reader' command when connecting to a feeding site this is the default. You will probably never have to change this setting.
fts_dird:\dnews\ftsSets the directory where the fts index files are being stored. (see the section on FTS for more details)
fts_expiretrueAutomatically set if fts_dir is true this tells dnews to write a list of all expired articles so that the fts system can remove them from the fts database.
groups_at20 2Sets when dnews should check for newly created news groups on the site that it is sucking news from. (not needed with an IHAVE feed where the control news items perform this function).
head_cache200How many item headers to keep in memory if you have memory to burn then a setting of 3000 will improve performance of some nntp commands
head_limit2000Limits the size of headers to keep in the cache the default is about 2000
hide_queuetrueStops the list of groups being sucked from being shown in the 'Downloading...' message.
history\dnews\spool Directory for history file this could grow to 40MB for a full feed (80 000 blocks)
history_min5 Pause if less than 'n' megabytes free
item_max100000 Sets the maximum size for any article that DNEWS can deal with the default is 1 000 000. A smaller number will save memory.
life30 Number of days to keep fetching a news group from the upstream server after someone reads an item in it.
list_min300How many groups need to be cached before the update method switches to the technique that works best for large numbers of groups.
live_nostreamtruestops live feeds from using the streaming extensions to send articles (this configures automatically normally)
log_inlinefalseStops logging of all the 'item ok' messages inside the dnews.log file
log_nntpfalseStops logging all the nntp commands sent to dnews.
logdir\dnews Directory to write log files
logdir_min5 Pause server if less than 5MB is left free on logging disk.
loglvlinfo One of: error warn info debug flush

There are four levels of error reporting "error" includes serious errors "warn" includes possible problems "info" includes general information messages and "debug" which is used for tracing bugs. "flush" is the same as debug but the log file is flushed with every single line written this is useful if the system is crashing but not providing a stack dump.

logusetrue Write a log of every item body read by users in the file used.log
mailsmtp This is only used on VMS systems. The name of the foreign email system on VMS typically in% or smtp% (don't include the %)
mail_fromsystem@your.domain Sets the from address used when dnews sends a report or email confirmation to a user this is somtimes needed by smtp servers which require a valid return address.
managersystem@here.this.place Email address of the news manager
max_out10000Limits the speed that any channel can read news articles in characters per second.
max_speed10000Limits the speed in characters per second that any channel can send news into dnews.
mynamenews.this.place Your full internet name if left out this may be picked up from the system. It's best to put it in by hand to make sure it's right. See path_stamp below.
nntp_blocktrue Sets all the sockets as blocking we don't know why you would want to do this.
nntp_feedernews.feeder.site The news site where you are going to SUCK a feed from. You need NNTP READER access to this site. If you are not sucking a feed then comment this line out. If you need to specify a PORT other than the default one (119) then add it on the end of the name e.g. feeder.site.edu:2021
nntp_logoffYou are not one of our users Specifies the message to give to users who are not granted connect access via access.conf.
nntp_passmysecretword Use this if the site you are sucking from requires a username and password to access it.
nntp_suck2 3 4second.feeder alt.irc.* myuser mypassword Get irc groups from another news server every 10 minutes the user/passwords are optional. (also see update_groups)
nntp_userjbloggs Use this if the site you are sucking from requires a username and password to access it.
nntp_xpostingtrueFor a posting feed this lets you add a header which will identify any user that attempts to post spam thru your server.
no_newnewstrueDisables the newnews command.
nocontroltruePrevents automatic creation of the 'control' news group.
nodnstrue Stops DNEWS from doing reverse DNS lookups every time a user connects you will have to put ip numbers into access.conf if you use this feature but if your DNS is faulty or slow then this can improve connection times.
nospacechecktrueStops dnews attempting to find out how much disk space is free. If this test is failing your server may pause itself so this setting gets it working again. This is quite safe as the test is only used as a backup check.
nostreamtrueTells dnews to pretend it cannot take a streaming incoming feed.
nwserverhostnameUse on netware servers if NetWin suggests it should be used.
open_blocktrueMakes nntp open calls block the news server
open_retry5How long to wait after failing to open an nntp connection before retrying .
orgNetWin LtdSets the name to be added to local posts which do not have the Organization: header set
org_replacetrueForces local posts to have your organization name even if the news posting software attempted to put some other name in.
out_size4000Sets the size of packets to be sent in one write operation to a socket.
path_stampnews.netwinsite.com The name used to stamp the path of messages that go thru your system this should be a fully qualified text name in lower case it need not have a genuine DNS entry.

Use this if your MYNAME parameter is a single word or a number. e .g.myname 22.33.23.4 path_stamp news.netwinsite.com. Ver 4

pidenttrueEnables pidentd checking (a way of authenticating users on multi user unix systems)
port119 The TCP port which DNEWS should listen for incoming connections on.
post_logtrueLogs every local message to a post.log file
post_routetrueEnables routing of incoming ihave messagse to outgoing posting feeds.
postonlytrue Stops items from being posted locally until the 'echo' back from the feeder site this gives a very clear confirmation that the message has been passed up stream. You cannot do this if you want to have local groups which are not on the site you are sucking from.
replicatetrueMakes dnews use the same item numbers for each item as are used on the server it's sucking from. The result is a second server that mirrors the first.
reportno Stops the daily email report from being sent to the manager after every expire run.
send_at*5 * Send messages upstream when the time matches this would do it every 10 minutes. e.g. at 1:05 1:15...
send_latertrue Stops DNEWS from attempting to send a post immediately useful if you are using a dial up connection. You can trigger this by hand using 'tellnews requeue'
sendmail/usr/ucb/sendmail Path to sendmail or equivalent software with switches. (only used on unix)
sort_chunk1000000Sets how many items to read in one go when re-sorting the master index setting this larger speeds up the sort but uses more memory. Ver 4
spool_freetrueThis tells dnews to chose which spool area based on the amount of actual free space rather than it's internal count of how much space it's allowed to use. Ver 4
spool

spool2

spool3

.......

spool8

spool f:\spool

spool2 g:\spool

spool3 h:\spool

spool4 i:\spool

Disk name (or logical) for news articles

For use with multiple drives (upto 8). Spool2 ... Spool8 are Version 4 commands.

Defines the disk areas that dnews can use to store it's buckets of news.

See diskspace above.

spool_min5 Pause if less than 'n' megabytes free
suck_batchn100Maximum number of items to get from one group before going onto the next group.
suck_batchsize1000000Maximum bytes to get from one group before going onto the next group. Use this to stop dnews spending all day on a binary group.
suck_requeuetrueTells dnews to reque a group that is not finished.
suck_maxold1000 Limits the maximum number of old items that dnews should suck from a group.
suck_streamtrueMakes dnews suck much faster don't set this unless you need to as it hurts the server you are sucking from.
suck_timeout300How many seconds a sucking feed can be idle before dnews should close the connection to try and get it un-stuck.
suck_uucp/uucp_in/*.bagDirectory and file names of incoming uucp files. Every minute dnews will scan this directory and read in any files it finds and then delete them. They should be in standard rnews format.
tcp_nodelaytrueSets the socket to send data immediately this may solve problems with bugy news readers. (e.g. Mosaic news reader)
timeout30 Define how long to wait after a channel becomes idle before disconnecting it the default is 120 minutes.
timezoneNZST Timezone hh:mm or timezone name Appendix B
too_old50 Refuse items that were created more than this many days in the past this prevents old items from being re-distributed when a machine dumps old news items onto the network.
update_at*0 *5 * Check for new items every time the minutes end in a 5 or a zero and at any hour e.g. every 5 minutes..
update_at2 3 410 *Update setting for second third and fourth feeds check less frequently every hour at 10 minutes past.
update_groups1 2 3*binary* 45 1 A way of getting specific groups more frequently only works with the main feed. For example get binary groups once a day at 1:45 this is in addition to the update_at setting not instead of.
uucp_crlftrueWrites uucp files as MSDOS text files instead of proper unix format.
workarea\dnews\spool Directory for work files (active.dat *.feed etc)
workarea_min5 If less than 'n' megabytes are free on this disk then pause the news system (stop accepting new articles)
xover_cache4000How many xover records to cache this is the best way to improve performance if you have spare memory set it as high as 15000.

[return to index]

access.conf

This file controls access to your system you can control access by IP name IP number or even by individual user using password protection.

<host>:<perm>:<user>:<pass>:<groups>

ParameterExample Description
<host>*.my.domain

161.29.4.*

A wildcard name or number this rule is only used if the connection matches this address
<perm>Read Post Ihave logoff Allow any combination of reading posting or ihave feeding from this site.
<user>chrisp Some news readers can send a user name if so then it will be matched here and the password can be checked.
<pass>rumplestiltskin The password in plain text.
<groups>* !*.erotic.* Allow access to groups matching this wild card.

Notes:

If <user> is "$lookup$" and <pass> is "$lookup$" then NT Authorization is used to lookup users via the nt database. Use a line like this in access.conf: your.ip.range.*:read post:$lookup$:$lookup$:*

Unlike other news implementations you can permit reading and posting to different combinations of news groups for example you may wish to allow students to read all groups but only post to local news groups.

The last matching line in the file is used. This file can be tricky to setup if it doesn't work as expected check that you have the entries in the right order so that the last matching ip name does what you want.

[return to index]

expire.conf

This file can be tricky to use remember order is significant!!

The expiration file is very different from INN and CNEWS implementations Rather than a simple set of rules the DNEWS expire rules match both news group names and current disk usage so as use increases the rules can become more strict.

In addition rules can specify minimum or maximum numbers of items this allows a single rule to apply to groups which get 20 items a day or 20 items a month.

Each line of the file must have the following format:

UPTO x GROUPS wild KEEP n DEFAULT n PURGE n MAXITEMS n MINITEMS n
SymbolExample Description
upto20% This rule applies if less than 20% of the disk space is full.
groupsalt.* comp.binaries.* This rule applies if the group matches this list
keep2 Keep items for this many days even if they have already expired. (From the expire date in the article header)
default10 Keep items this long and then delete unless the expiration date says to keep it longer.
purge35 Delete after this many days even if the expire header says to keep it longer. This should be set long enough to keep FAQ's and no longer to prevent abuse.
maxitems1000 Delete old items if the group has more than this many items in it. Good for controlling high volume groups.
minitems10 Don't apply any other rules if the group has less than this many items in it this is good for making 'low volume' groups useable.

Example:

# Default rule
upto 10000% groups * keep 0 default 1 purge 10 maxitems 0 minitems 0
# Start to panic a little
upto 90% groups * keep 0 default 2 purge 33 maxitems 100 minitems  0
# Normal operation
put your meaningfull settings here.
upto 60% groups * keep 2 default 5 purge 33 maxitems 1000 minitems  10
upto 60% groups comp.* keep 2 default 10 purge 33 maxitems 2000 minitems  10
# Lots of space
just stop things getting silly (e.g. 10000 items in...)
upto 30% groups * keep 2 default 17 purge 33 maxitems 3000 minitems  100
upto 20% groups * maxitems 2000
# Keep any local groups forever (this line must be last to work)
upto 1000% groups local.* keep 99999 minitems 10000

If disk use is 58% then the rules that will be applied are the upto 60% rules once the useage goes past the upto value of a rule the rule is no longer matched on. DNEWS will use the last rule it finds that matches searching from the top down. The upto 60% rules comprise of one rule for all groups and one rule for the comp.* hierachy the specific rule must be after the general rule or it wil never be matched. DNEWS will apply the rule like this if the item has an expire header DNEWS will apply the keep rule and keep the item for 2 days even if the expire header says not to. The default rule is the rule that gets applied to all of the articles DNEWS will expire an item that it has had for more than this many days unless the expire header says not to for the comp.* groups this is 10 days for all of the other groups 5 days is applied. The purge setting removes anything that has been on the server for this many days even if the expire header says not to this setting is used for removing periodic posts such as FAQ's. The maxitem and minitem settings overide the other settings and will keep or remove items even if the other rules say not to. There are cases where the maximum and minimum counts may seem to not be applied this is usually due to crossposts.

[return to index]

newsfeeds.conf

This file controls feeds out of DNEWS to other sites it also controls the feed 'into' DNEWS via the same mechanism.

SymbolExample Description
siteme

$NNTP_FEEDER

my.leaf.site

The name of the site to feed usually it's full internet name. The 'me' feed must be the first one in the file.
builtin(flag no value) Specifies the builtin feeding mechanism should be used. This is only intended for sending local postings up stream it is not fast enough to send a full feed on to another site.
posting(flag no value) Specifies that the nntp 'post' command should be used to send items upstream. This gives your feeder site the job of deciding how to email postings to moderated groups.
excludefeeder.real.name feeder.third.name Use this if your feeder site is 'name stamping' articles with a name other than it's ip name. This MUST be right or DNEWS will attempt to echo everything it gets back to the feeder site. (check the path header)
smtp(flag no value) Specifies this is not really a feed but on a vax it is actually the email queue used to send articles to moderated groups this is only needed on VMS and then only if you are not using the 'POSTING' mechanism to feed upstream.
maxcross50 Throw away items which are crossposted to more than '50' news groups. This is good for preventing 'spamming' which is where people post adverts and money making schemes to inappropriate news groups.
maxsize100000 Limit items to 100K the default is 1MB if you want to make it larger than 1MB then you must also increase ITEM_MAX in DNEWS.CONF. Personally I think 100K is too big ftp sites and web should be used for larger items. Dnews will still download the items it just doesn't store them if they are larger than this setting.

Setting your EXCLUDE parameter correctly:

In newsfeeds.conf you must set your exclude parameter to stop DNEWS from sending incoming messages back to your feed site. Lets assume your feed site is called fastfeed.world.net. You need to find out what it stamps onto news articles to do this read a news article (and show all headers so you can see the PATH header)

e.g.

Newsgroups: alt.winsock.trumpet
Subject: Sportster Si does not seem to work with TRUMPET
Date: Thu
26 Oct 1995 22:44:08
Organization: Personal
Lines: 8
Message-ID: <moleski.2.0016BD02@sasknet.sk.ca>
NNTP-Posting-Host: eagle28.sasknet.sk.ca
X-Newsreader: Trumpet for Windows [Version 1.0 Rev A]
Path: fast1.world.net!world.net!news.icix.net!esol.intermedia.com

Look thru the path left to right and find the first or most generic name for your feed site in this case 'world.net' looks like the best name to choose.

So here is the entry you should put in newsfeeds.conf you may need the builtin and posting settings as well but that is up to you.


site fastfeed.world.net groups * exclude world.net builtin

You may need to delete your .feed and .sending files from the spool directory to clear out the backlog of articles queued.

[return to index]

moderators.conf

You won't need to touch this file unless you want to add a local group and make it moderated. In that case add your groups at the top of the file as the first match is used.

Format:

newsgroup:email-address

If the email address contains a "%s" then that is replaced by the news group name with the dots changed into dashes.

Example:

local.gossip.moderated:chrisp@marc.cri.nz
gnu.*:%s@tut.cis.ohio-state.edu
*:%s@uunet.uu.net
[return to index]

control.conf

This file controls the action taken when control messages are received. This is the secret to the administration of news. Control messages are just like normal Usenet news articles but they are read by your news server and make it automatically do things like adding and deleting news groups. This may seem invasive but with dozens of news groups being created and deleted every day you really don't want to be doing this yourself.

See the section on Adding PGP authentication to control.conf if you are concerned about secuirty issues involved with control messages. However there is not too much need for concern DNEWS does not actually remove anything if a news group is deleted so it can always be recreated if necessary.

Cancel messages are not processed by this file for performance reasons they are dealt with differently.

The following control messages are checked by this file:

However only the following three are implemented.

Blanks are not allowed e.g. :: would be misinterpreted.

For each type of control message you can specify based on who sent it whether or not to do what the message says just log it or email the message to the news administrator. The log file that DNEWS creates can be used to action the control messages at a later date as it contains valid commands.

All control messages are logged to 'control_(message).(action)'

Format:

message-type:email-address:action
message-type    rmgroup
newgroup
version
email-address   A wildcard email address
e.g. news@trusted.site
action          "drop"
"log"
"mail"
"doit"
"doit
mail"
"doit
pgp"

Example:

all:*:*:mail
ihave:*:*:drop
sendme:*:*:drop
sendsys:*:*:drop
senduuname:*:*:drop
newgroup:tale@*.uu.net:comp.* misc.* news.* rec.* sci.* soc.* talk.*:doit
rmgroup:tale@*.uu.net:comp.* misc.* news.* rec.* sci.* soc.* talk.*:doit

[return to index]

Chapter 10 Dnewsweb - A Web Based News Server Interface

Upgrading DNEWSWEB

To upgrade you simply copy the dnewsweb.exe file into your CGI-BIN directory on your web server. If you are not using top and end files then you should remove the 'headers' setting from dnewsweb.ini

Installing DNEWSWEB

DNEWSWEB is a web to news gateway it lets you merge your web pages and your news groups.

You must install it on your WEB server in a directory that you have already setup on your server as containing CGI programs you must read your server documentation if you don't currently know how to setup a cgi program typically you would copy the files to a directory like:

SERVER_ROOT\cgi-bin\

Two files should be coppied to this directory

copy \dnews\dnewsweb.exe \SERVER_ROOT\cgi-bin

copy \dnews\dnewsweb.ini \SERVER_ROOT\cgi-bin

In some instances you will have to place the .ini file in a different directory this is dependent on your server software sometimes the 'SERVER_ROOT' is the right directory. (Dnewsweb will give an error if it can't find this file in which case just try moving it somewhere else)

Now edit dnewsweb.ini and define three things

notepad \SERVER_ROOT\cgi-bin\dnewsweb.ini

The name of your DNEWS news server

News groups to appear on the front page

News groups that you are allowing people to access.

WARNING: if you don't limit the news groups people can access then you may find everyone in the world reading news via your web pages this may make your site more popular than you would like :-)

Then try it out point your web server at the address:

(unix) http://your.ip.name/cgi-bin/dnewsweb

(nt) http://your.ip.name/cgi-bin/dnewsweb.exe

It will run with other news servers however it is much faster with DNEWS and the 'list all newsgroups' function will only work with DNEWS.

HELP It asked me where to save dnewsweb.exe ????

This means you have failed to tell your web server that the directory cgi-bin/ contains CGI programs and not WEB pages. Go back to your web server administration window and look for help on CGI directories.

(This is the single most common question we get !!)

Header and Footer Files

You can replace the top and end of each page that dnewsweb displays you may wish to do this to add your own graphics and backgrounds etc.

In dnewsweb.ini define

headers d:\dnewsweb

Then in this directory place files with the following names:

 

File Names Description

main.top main.end List of main news groups

list.top list.end List of all news groups

article.top article.end Individual articles

search.top search.end Full Text Search output

fsearch.top fsearch.end Full Text Search Form

ffind.top ffind.end Search headers in a news group Form

find.top find.end Search headers in a news group

fpost.top fpost.end Posting Form

dir.top dir.end Directory of items in a group

Each .top file should contain somthing like:

<html><head><title>$title$</title>

</head><body>

The $title$ will be replaced with the group name etc as dnewsweb would normally display it.

Each .end file should contain any footer information you wish to add.

The end tags "</body> </html>" are appended so you do not need to include them in your .end file.

Full text search engine

To enable the full text searching button you need to add

search true

to dnewsweb.ini (as well as installing the xmit process to build the indexes see install.txt from dnews for details)
[return to index]


Chapter 1. Dnews and Usenet etc etc ...

Include in here the stuff from Chris' presentation re Usenet dnews architecture etc diagrams...

DNEWS is a dynamic news server system it is not a news reader but rather it is the server which a news reader reads news from.

DNEWS Feature list:

Where to get Dnews

Dnews software is available from our website from two directories the top level is where we put the released versions of Dnews and when we release a new version this is where it will appear.

ftp://ftp.netwinsite.com/pub/netwinsite/dnews/

The other directory is the beta directory this contains beta versions which have new features bug fixes etc. these versions are available for use at your own risk or on our advice.

ftp://ftp.netwinsite.com/pub/netwinsite/dnews/beta (the latest beta versions of dnews)

Error Messages

Common error messages and what they mean.

411 License key expired sorry

This error is returned to users when your license key has expired. NetWin are always happy to provide a temporary license key to get things going until the 'paper work' can be sorted out or until you have finished your testing process. netwin@netwinsite.com

411 no such news group

This means the news group does not exist in the active.dat file on this server.

411 no permission to read that group

This means the users does not have permission to read the group see the file access.conf

411 group did exist but has been deleted

This means the group has been deleted with the 'rmgroup' tellnews command it can be undeleted with tellnews if necessary.

Read error on group %d:%d offset %d wanted %d got %d

This usually means the index file '.idx' for the specified group was corrupt. An addlost is probably a good idea if these happen too often.

Error reading index file {%s}

Ignore this error DNEWS will often look for files that don't exist this is considered normal :-)

Syntax error in expire.conf
expire: syntax error in expire.conf line %d
Syntax error in expire.conf {%s}

This means one of the lines in your expire.conf file did not make sense check the file carefully and make sure you are editing it with a 'text' editor.

active.times is too big please use command: tellnews fixtimes -100

This means the active.times file which grows slightly every day is getting a bit too large you must issue the suggested command you need to do this every 6 months or so.

Discarding incoming data due to big error above %d

This may occur if you run out of space in the workarea directory basically look above this error to find the real cause of the problem.

nntp: Socket closed errno=%d

This means an nntp connection from a reader or another server closed in an unusual way normally this can be ignored. You can lookup the errno in a unix errno.h file to see if that gives more help but it is unlikely to.

socket_read failed

Often this occurs when a news reader closes a connection unexpectedly it can usually be ignored.

Socket newchan accept() call failed

This means someone tried to connect to your server but when dnews tried to 'accept' the connection the user had gone or somthing prevented the connection being accepted this is an unusual error and may be significant if you are having other problems.

socket_write error

Ignore this it's usually just caused by a connection closing unexpectedly.

wsock: (%s) The virtual circuit was reset by the remote side.

This usually means the other end closed the connection.

wsock: (%s) Connection Refused

This usually occurs when you try to connect to a news server that isn't running at all it doesn't mean that you were refused access by the access.conf file.

wsock: (%s) Address in use

This often means you have some other news server or port mapper running which has a hold on port 119

last error: NNTP open of {%s} failed check dnews.log

This error can often be ignored but it implies that the sucking feed may be down briefly if you stop getting new news then this would indicate something is wrong with the place you are sucking news from.

suck: BAD status code returned {%s} {%s} {%s}

Read the error message inide the brackets carefully this error probably came from the site you are sucking from not dnews itself so if the error in the brackets sais 'you have no permission to talk' then the problem is that the place you are trying to suck from has not given you permission to talk.

 

DNEWS FAQ

(Frequently Asked Questions )

1.0 General Information

2.0 General "How do I" questions.

3.0 Common Configuration Questions

4.0 I did something stupid can I fix it?

5.0 Errors & Problems

6.0 I still can't figure it out how can I get help?


1.0 General Information

1.1 The FTP site

You can FTP the latest copy of the files from our main ftp site their are binaries for common platforms (Windows NT (intel & Alpha) Linux Solaris VAXVMS ALPHAVMS.

ftp ftp.netwinsite.com
cd pub/netwinsite/dnews
binary
get dnews26o.exe  (Windows NT version 2.6o)

This is a self extracting archive that means you just run dnews_nt.exe and it will extract itself e.g.

dnews26o.exe   (extracts the new files to \tmp)
cd \tmp
install.cmd  (or newver.cmd if you already have dnews      installed)

Use the following references to see our web pages and ftp area.

http://netwinsite.com/
ftp://ftp.netwinsite.com/pub/netwinsite/dnews

Return to the Index

1.2 Why is a sucking feed so great?

A sucking feed has several advantages not all obvious.

Disk space and line usage

This is the most obvious advantage you may only end up getting a couple of hundred groups possibly saving 80% of your disk space and 80% of your network bandwidth. A 64Kbps line isn't enough to take a full news feed but with a sucking feed it is easily enough to give the 'impression' of a full news feed.

More items in 'low use' groups

More items in 'read' groups: As disk space is saved it becomes very easy to keep 1 or 2 weeks of news (or more) in the groups people actually want most full feed systems can only store a few days of news in each group.

Efficient file IO

As items are fetched for one group at a time most file IO is to a single open file this is a lot more efficient that the standard news system of getting items in a 'random' order. This efficiency is more significant on operating systems with slower file systems e.g. Windows NT and VMS.

Low impact on upstream site

DNEWS does not use the 'newnews' command which all news managers hate it uses low impact commands to find the new messages and retrieve them without accessing the history database on the upstream server. As so much fewer articles are fetched the overall load is greatly reduced. The exact load depends on how many groups are being sucked.

Easy configuration on upstream site

On the upstream site instead of setting up and managing a feed something which is difficult to do and tends to require almost constant maintenance all that is required is to give your system simple 'news reader' access.

Return to the Index

1.3 Where are the log files?

Look in dnews.conf for the symbol LOG_DIR this will tell you which directory to find them in there are three files:

dnews.log       The main log file
dnews.in        A one line log of each message that came in
dnews.out       A one line log of each item dnews sent out.

The file dnews.log contains .in and .out messages as well. By default the logging level is set to INFO you can set it in dnews.conf to one of error warn info debug. E.g.:

loglvl warn

In debug mode all error messages are sent to the log file in info mode only error warn and info messages are printed. In warn mode only error and warn messages are sent and in error mode only serious errors are printed to the log file.

Every night or if the log files get more than DEBUG_ROTATE lines in them the files are rotated like this:

dnews.log -> dnews.log2 ->dnews.log3 ->dnews.log4 ->(deleted)

Return to the Index

1.4 Where is dnews.conf?

This is the main configuration file and you can't move it from it's built in directory which is:

Windows NT      /winnt35/system32/dnews.conf (system dir)
Unix            /etc/dnews.conf

All other configuration files will be in the config directory defined in dnews.conf

Return to the Index


2.0 General "How do I" questions.

2.1 How do I limit user access to the server

You can limit user access to the server by username and password and by group to do this you need to edit access.conf. This file is both tricky in it's syntax and order sensitive but quite powerful.

Note that the netscape news reader does not currently implement usernames and passwords so you will need to make your users use a proper news reader :-)

Here is a simple access.conf where you only allow users in your domain with ip names and you only let them connect using usernames and passwords:

# By default don't let anyone even connect
*:::!*
# Let people in my domain connect long enough to send a username/password
pc*.here.edu:read:::!*
# Here are the individual users and their passwords
pc*.here.edu:read
post:joe:joespassword:*
pc*.here.edu:read
post:alice:alicespassword:*
pc*.here.edu:read
post:jack:jackspassword:*

Here is a second example where we want to stop joe from posting to the group local.rules but he can read it ok still.

# By default don't let anyone even connect
*:logoff::!*
# Let people in my domain connect long enough to send a username/password
pc*.here.edu:read:::!*
# Here are the individual users and their passwords
pc*.here.edu:read:joe:joespassword:*
pc*.here.edu:read
post:joe:joespassword:*
!local.rules
# And don't let anyone from the finance section even connect
*finance.here.edu:logoff:::*

Return to the Index

2.2 How do I set my local groups so they won't expire

At the end of your expire.conf file add a line like this:

upto 1000% groups local.* keep 999 minitems 10000 maxitems 100000

This will keep all items in the local group for 999 days and even then will keep at least 10000 items but will expire items if more than 100000 items are in the group.

Return to the Index

2.3 Creating local news groups

If you wish to have local groups on your server that are not on the server you 'suck' from then you should remove the 'POSTING' flag from the feed to your feeder site (in newsfeeds.conf). But only if you can get ihave (or transfer permission) to send articles up stream rather than simple 'posting' access.

You can still have local groups if you can't get ihave access it's just not the 'recommended' setup.

You can add groups with the TELLNEWS NEWGROUP command e.g.

tellnews newgroup here.chat y myname Local group
here.chat = name of group
y = Normal group (m = moderated)
myname = A record of who created the group (one word)
Local group = description for active.names file.

To stop your local group from being sent up stream you would change the line in your newsfeeds.conf file:

groups *
!here.chat
!here.secret.*

NOTE: KNOWN BUG with version 2.0 items will still be sent upstream if you have left the 'POSTING' keyword in your feed. This will probably be fixed in a future version. (ofcourse this isn't a major problem unless the items are very secret as your upstream site will probably just reject them anyway)

Return to the Index

2.4 Can I restore some news files (*.itm *.idx) from tape?

You have restored part of a database
to merge this new stuff into the indexes use:
tellnews addlost

Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news.

Return to the Index

2.5 How do I move the spool area to a bigger disk

First shutdown DNEWS:

tellnews exit

Then move the entire SPOOL tree structure from it's current place to it's new place for example on NT you might use file manager to move

c:\dnews\spool...             to      e:\dnews\spool...

Change the definition of SPOOL in DNEWS.CONF

NOTEPAD \winnt35\system32\dnews.conf
spool   e:\dnews\spool

Restart DNEWS:

(NT)    Control Panel
Services
Start
(UNIX)  su root; dnews_start.sh

Return to the Index

2.6 How do I make the updates run only at night

Updates for a sucking feed normal run once every ten minutes this is controlled in DNEWS.CONF with the command

update_at *5 *

This "*5" tells dnews to do an update whenever the minutes end in a digit '5' and the second "*" tells it to do updates during any hour.

To do one update a night at 5 past 2 in the morning you would use:

update_at 5 2

To do updates every 10 minutes from 6p.m. to 8a.m. you would use:

update_at *5 18-8

Return to the Index

2.7 How do I test the server using TELNET

NNTP is a text based protocol this means you can test it and even read news without any software just by using telnet and typing in commands by hand this is not a seriously good way to read news but it is a very good way to test your own server or your feeding server:

NT      Accessories
Telnet
Connect
Remote Service
Enter the feeder.site and under PORT type in NNTP
Under Terminal Preferences turn ECHO on
Press Control J to end lines instead of ENTER
VMS     $ telnet feeder.site /port=119
UNIX    > telnet feeder.site 119
Type in this sequence of commands to read an article by hand:
group comp.os.vms       (select a group)
head                    (display the header)
body                    (display the body)
next                    (next item please)
help                    (show valid commands)
quit                    (exit the connection)

Return to the Index

2.8 How do I suck from a two sites at once

Use the alternate suck names NNTP_SUCK2 NNTP_SUCK3 NNTP_SUCK4

Put these in dnews.conf followed by the list of news groups that should be sucked from these alternate sites. You should use the NNTP_FEEDER variable (suck1) for the largest number of groups.

nntp_feeder news.upstream.site
nntp_suck2 other.news.site chat.local.*

Alternatively you can take a feed and suck from another host at the same time.

Your list of groups will be based on the NNTP_FEEDER host other groups must be added using the tellnews newgroup command.

To do this just setup the system to suck as usual but then add a line to access.conf to allow your secondary feed to come in e.g.:

other.feed.machine:ihave:::*

Return to the Index

2.9 How can I cut out spamming (cross posts)

Spamming is where people cross post to hundreds of news groups with information which is not relevant to the content of the news group. e.g. advertising pyramid schemes etc...

Cross posting is a neat idea but enormously abused mechanism you can limit the abuse by rejecting items that are crossposted to more than 'n' groups a value of 5-10 would seem reasonable to me. The default is 100 for DNEWS.

You can chage the limit in newsfeeds.conf under the ME settings e.g.

site me
groups *
!*binary*
maxcross 4

Return to the Index

2.10 How can I make XOVER faster?

By default DNEWS caches the last 200 item headers that it has read if you have spare memory then it is well worth increasing this value: The upper limit is 9000 at approx 3K per header this would use about 27MB but a value of 2000 would use 6MB which most systems could easily deal with. In DNEWS.CONF

HEAD_CACHE 2000

Return to the Index

2.11 How do I rename a volume on UNIX

You may wish to rename a current volume /var/spool/news to be /var/spool/dnews to do this:

cd /var/spool/news
mkdir OLD
mv * OLD
umount /var/spool/news
emacs /etc/vfstab (or /etc/fstab)
(change /news into /dnews)
mount /var/spool/dnews

When you are happy with the new system or you need the space:

rm -r /var/spool/dnews/OLD

Return to the Index


3.0 Common Configuration Questions

3.1 How do I setup DNEWS to take in a 'normal' NNTP - IHAVE feed

In access.conf give your feed site access to send you ihave messages:

my.feeder.site.com:ihave:::*

In dnews.conf comment out the nntp_feeder definition.

In newsfeeds.conf replace:

site $NNTP_FEEDER
groups *
builtin
posting

With:

site my.feeder.site
groups *
builtin

And in dnews.conf add a line:

cachedall true

You need to FTP an active file from your provider if he won't supply you one then you can get one of the NetWin FTP site but it is better to get a local one. Copy this file to your spool directory:

(Stop DNEWS first)
copy active_from_provider.txt /dnews/spool/active.dat

Now read the FAQ section entitled "3.6 How do I set my EXCLUDE parameter correctly?"

Here is an example of a full feed from UUNET. Note the exclude parameter is essential.

-------dnews.conf---------
In  dnews.conf  comment out nntp_feeder by putting a HASH in
front of the line
and add cachedall true
e.g.
# nntp_feeder   your.feeder.ohio.edu
cachedall true
------newsfeeds.conf cut here ----------
site me
groups *
site news.uu.net
groups *
!local.*
builtin
exclude uunet
site YOUR.MAIL.GATEWAY
smtp
------access.conf cut here and change your domain and ipnumbers ------
# Don't let the world connect :-)
*:logoff:::*
# Do let your domain and ip range connect    **** CHANGE THIS BIT ***
*.your.domain:Read
Post:::*
161.232.32.*:read
post:::*
# Do let your news feed from UUNET send you items
news*.uu.net:ihave:::*

Return to the Index

3.2 How do I setup DNEWS to take an ihave feed from 4 machines and send back to another.

Lets assume you have been told you will get a feed from any or all of:

news1.xxx.net
news2.xxx.net
news3.xxx.net
news4.xxx.net

And you have also been told to send your feed back to

newsup.xxx.net

And after examining the PATH header in a news item from your upstream feed you have seen that they stamp all incoming messages with the name 'news_stamp.xxx.net'

Then you would setup access.conf like this:

news1.xxx.net:ihave:::*
news2.xxx.net:ihave:::*
news3.xxx.net:ihave:::*
news4.xxx.net:ihave:::*

And you would setup newsfeeds.conf like this:

site me
groups *
site newsup.xxx.net
groups *
builtin
exclude news_stamp.xxx.net
site your.mail.gatewway
smtp

Return to the Index

3.3 How do I setup XMIT to send to other sites

If you have more than one large feed coming into your site then you should use XMIT to send to all sites and NOT use the builtin feeding system at all (see the keyword builtin above remove it!!) The builtin feeder will not run fast enough to 'pass through' large feeds so you must use the XMIT feeder in all directions. (Leaf sites can still use a sucking feed quite safely as this is also very efficient)

The XMIT feeder is a high speed auto configuring streaming feeder with built in scheduling so you don't need to use CRON or anything else tricky like that.

First add entries to your NEWSFEEDS.CONF file this creates the .feed file that XMIT will use:

NOTEPAD newsfeeds.conf
site other.site.here.com
groups *

Now modify your startup procedure to run the xmit job for each feed you have defined in newsfeeds.conf

Adding XMIT to Windows NT Services (startup)

On windows NT you need to setup each xmit job as a service to do this use the program ADDSVC.EXE which came with the dnews distribution to add a service for each outgoing feed e.g. (notepad setupsvc.cmd and then run it to create the feeds)

addsvc -add dxmit1 xmit.exe "dxmit1 site.name.edu -at *5 *"
addsvc -add dxmit2 xmit.exe "dxmit2 site2.name.edu -at *5 *"

Then from CONTROL PANEL SERVICES and start the feed(s).

Adding XMIT to UNIX startup

Add lines to your dnews_start.sh procedure e.g.

cd /usr/local/dnews
./xmit send.to.site.name -at "*2" "*"
./xmit send.other.site.name -at "*7" "*"

Adding XMIT to VMS startup

Modify RUNXMIT.COM and add it to dnews_start.com

Return to the Index

3.4 What files can I copy from an INN or CNEWS system

This is not necessary but if you are replacing a system then there are some files that you may want to preserve so that news.rc files are not 'too' confused by the new system.

You can safely copy these files:

active          -> active.dat
active.times    -> active.times
newsgroups      -> active.names  (may be changed in future)

If you copy the active file you must reset DNEWS with this special manager command:

tellnews resetall

Return to the Index

3.5 How do I setup a news gateway for my LAN

With DNEWS a modem and Windows NT (or 95) you can give all the users on a LAN internet news access (to read and post) with only a single dial in modem to any local internet provider. You may have to trigger the updates by hand to avoid leaving the modem connected 24 hours a day.

To do this install a modem and ethernet card in a pc then configure the pc for tcpip access to your local lan using a local ip number (you can make this up) and then configure it again for modem internet access to your local provider (probably using an ip address provided via ppp).

Here are some example settings if your LAN is not on the internet.

Ethernet Adapter:
IP Address              1.1.1.1 (anything just give each pc a different number)
Subnet Mask             255.255.255.0
Gateway                 (leave blank)
DNS                     (Leave blank)
Modem TCPIP Adapter:
IP Address              (leave blank
provider will supply)
Subnet Mask             (blank)
Gateway                 (blank)
DNS                     1.2.3.4 (You MUST get this from your provider)

Then for myname in dnews.conf you would use 1.1.1.1 and from other PC's on your lan set their news server to be 1.1.1.1

Return to the Index

3.6 How do I set my EXCLUDE parameter correctly?

In newsfeeds.conf you must set your exclude parameter to stop DNEWS from sending incoming messages back to your feed site. Lets assume your feed site is called fastfeed.world.net. You need to find out what it stamps onto news articles to do this read a news article (and show all headers so you can see the PATH header) e.g.

Newsgroups: alt.winsock.trumpet
Subject: Sportster Si does not seem to work with TRUMPET
Date: Thu
26 Oct 1995 22:44:08
Organization: Personal
Lines: 8
Message-ID: <moleski.2.0016BD02@sasknet.sk.ca>
NNTP-Posting-Host: eagle28.sasknet.sk.ca
X-Newsreader: Trumpet for Windows [Version 1.0 Rev A]
Path: fast1.world.net!world.net!news.icix.net!esol.intermedia.com

Look thru the path left to right and find the first or most generic name for your feed site in this case 'world.net' looks like the best name to choose. So here is the entry you should put in newsfeeds.conf you may need the builtin and posting settings as well but that is up to you.

site fastfeed.world.net
groups *
exclude world.net
builtin

You may need to delete your .feed and .sending files from the spool directory to clear out the backlog of articles queued.

Return to the Index

3.7 How do I setup DNEWS for local (intranet) use?

This is fairly simple to achieve install DNEWS normally then in dnews.conf set

cachedall true

Don't define the 'nntp_feeder' variable at all remove the word 'posting' from newsfeeds.conf then start the server.

Now create your local groups try to pick names that will never conflict with internet names if you decide to join up at a later date to achieve this add your companys name to the start of each group name (or an abbreviation of it)

tellnews newgroup netwin.chatting y Group for chatting about things
tellnews newgroup netwin.schedule y For scheduling meetings
tellnews newgroup netwin.products y Product sheets and info for sales staff

Done!

Return to the Index


4.0 I did something stupid can I fix it?

4.1 I deleted some item files by hand is that ok?

This is like editing a database with a binary editor and removing a few records by hand it is not a good idea :-(

However DNEWS is written for people just like you the database will correct itself with a little luck. :-)

Please do not edit an item file and remove individual items or change them this will corrupt the database and the items will simply dissappear.

Return to the Index

4.2 I deleted the history.dat file is that ok?

No it's not. You need to do this type in:

tellnews addlost

And cross your fingers and wait until the addlost finishes (it could take a few hours on a full feed system)

Return to the Index


5.0 Errors & Problems

5.1 Large messages & Memory Use.

Unless you have infinite memory on your system then it is necessary to limit the size of messages that can be processed. This also saves a lot of wasted bandwidth and disk space for you and every other news system. DNEWS limits messages to 1MB by default (a fairly generous limit considering many news readers still die horribly with messages greater than 64K and anything bigger than this should not be sent via news but should be put on an FTP/WEB site and a pointer to it should be sent)

To deal with messages of size 1MB dnews needs about 6MB of memory so if you are short of memory you may want to reduce this. To limit messages to 100k you would add a line in DNEWS.CONF

ITEM_MAX 100000

(this setting is not dynamic you must restart DNEWS)

Return to the Index

5.2 I'm not getting any binary groups why not?

The default configuration file (newsfeeds.conf) that comes with DNEWS excludes all binary groups. In my opinion these groups should not exist they are a left over from the days before FTP and WEB and even EMAIL so that news was the only option now it is much more appropriate to post pointers to large files rather than the file itself.

Anyway you or your users demand it to fix the problem edit your newsfeeds.conf file and remove the !*binary* from the 'ME' feed.

Then use the commands:

tellnews undelete *binary*
tellnews matchfeed

Return to the Index

5.3 My news reader says the newsgroups don't exist?

Check access.conf you might have access to get in but no access to read the news groups so they will appear empty or give 'no such newsgroup' errors.

Return to the Index

5.4 I don't know if posts are working?

There are two main reasons for posts to fail:

  1. You've messed up your newsfeeds.conf file.
  2. Your provider is not allowing you to send messages via 'ihave'

Follow these instructions to figure out where the error is:

Post an item to some local TEST news group e.g. nz.test then check the file DNEWS.OUT this should give one line per message sent 'out' of your system if this file is empty or gives an error first consider if the error is from DNEWS or if it is from the site you are sending items to. Second look in dnews.log search for "out:" once you've found the failure message you will probably find more clues just above or below it as to what went wrong.

Return to the Index

5.5 I'm still not convinced my posts are getting to the other side of the planet?

Post an item to alt.test you should receive 10-15 messages from news robots all over the world.

Return to the Index

5.6 I have an old fashioned feed coming in but I'm not getting any items?

There are two main reasons for incoming feeds to fail:

  1. You've messed up your access.conf file (or your DNS does not work)
  2. Your provider has not successfully setup the feed.

In case 1) first try letting everyone in by putting a wildcard access line at the bottom of access.conf

*:read
post
ihave:::*
tellnews reload

Then check the log files or use tellnews status and you will see the name and number of any machine sending you IHAVE messages (in the list of connected channels) put this name in access.conf.

If tellnews status shows the number twice but no name then your DNS is not working use numbers in the access.conf file instead or fix your DNS.

Return to the Index

5.7 What should I do now that I've deleted history.dat

First I advise you not to play with guns because you are in danger of shooting yourself in the foot.

Before restarting dnews delete history.idx and history.dat and then start dnews and type:

tellnews addlost

Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news.

Return to the Index

5.8 Netscape shows items but I can't read them why?

This means the item is missing from the history database this should never happen but since it has try this command:

tellnews addlost

Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news.

Return to the Index

5.9 Why doesn't the status report match actual diskspace used?

DNEWS maintains an internal count of free disk space this can be wrong or possibly your setting for 'diskspace' in dnews.conf could be wrong. Decide which is wrong either fix the number in dnews.conf or use the 'setused' command to change the internal count e.g. to set the used value to 950MB type in:

tellnews setused 950

Then wait 2 minutes and do another:

tellnews status

Return to the Index

5.10 DNEWS goes very slow when users connect

Check your DNS is working try putting

nodns true

In dnews.conf and fix access.conf to use numbers then type:

tellnews reload

Then see if that speeds things up if it does then your DNS is faulty.

Return to the Index

5.11 My system runs out of memory?

First check your pagefile settings for a full feed you will need about 120MB of pagefile on Windows NT.

See the section on large news messages.

Send the output of tellnews status to the mailing list or NetWin and ask for comments.

Return to the Index

5.12 DNEWS goes very slow

Check your DNS is working.

Type in:

tellnews stats

(not status)

This gives a listing of NNTP commands and how long dnews is spending on each one this will show if one command is slow and also if one command is being used more than is reasonable. Also consider adding memory.

Send the output of tellnews stats and status to the mailing list or NetWin for further suggestions.

Consider installing more memory or a faster disk drive.

If the xover command is using most of the time then consider increasing the buffering of headers. (see dnews.conf options)

Return to the Index

5.13 DNEWS locks up for brief periods (or permanently)

Panic!

Check the log file dnews.log work out from the time stamps what dnews was doing when it locked up.

This can happen if you run an expire while lots of people are trying to use the system or while an ihave feed is sending stuff in.

Use the tellnews stats command this will also point to commands which are taking too long to process for some reason.

Check your DNS and also that site you are sucking from is working and it's news server is working either of these can cause a problem. If the site you suck from is down for a few hours you may have to remove the feeder line from dnews.conf until it is up again. (normally you won't have to do this it's just a thing to watch for)

Return to the Index

5.14 DNEWS Crashes

Check the end of the file dnews.log the only partly acceptable reason for a crash is if DNEWS runs out of memory if the errors indicate this then see the section on memory problems above.

If it gives an error 'signal xxx causing exit' then set the debugging level to 'debug' if the log files just ends suddenly then set the debug level to 'flush'

loglvl debug           (or flush)
debug_rotate 3000

Now next time it crashes email the last few hundred lines of dnews.log to NetWin. We really don't accept crashes as acceptable behaviour and will go to some extreme lengths to fix this sort of problem.

Instructions for getting a stack trace on NT

If dnews crashes on NT the most useful file you can send us (with the end of dnews.log) is the file:

/winnt/drwtsn32.log

This will help us find the problem and probably get you a fix within a few days.

Instructions for getting a stack back trace on solaris / unix

Add to dnews.conf 
	atexit_disable true
(wait for the next crash)
cd /usr/local/dnews
gdb dnews
core core
bt
(send the output of this to netwin)
Return to the Index 

5.15 TELLNEWS says Can't BIND?

Can't BIND and tellnews says "I can't talk to you" when you telnet to the port.

On LINUX the slackware distribution comes with a dummy nntp daemon installed you must remove this so dnews can bind to the socket.

edit /etc/inetd.conf
(comment out the NNTP line)
kill -HUP <pid of inetd process>

Return to the Index

5.16 TELLNEWS fails or locks ups

The DNEWS process may take 1-2 minutes to initialize so wait a bit before panicking.

Check if the process is running

ps -axl | grep dnews    (linux)
ps -ef | grep dnews     (SunOS)

Check the log files

more /usr/local/dnews/dnews.log

On UNIX if it gets started up with the wrong PID you will get all sorts of protection problems all the files in:

/var/spool/dnews/...

(including var/spool/dnews)

must be owned by the same user normally 'news'. Also all the files in:

/var/local/dnews/*

Also on UNIX it is possible to start dnews twice this can cause real disasters always check with ps and by looking at the log before restarting DNEWS.

Return to the Index


5.161 TELLNEWS gives unusual error message

On vms if the symbol tellnews is not defined it will try and run telnet instead this can be confusing. On Windows NT if you left the disk name out of the config file and you run the tellnews command from a different default disk then it won't be able to find all the files it needs to work.

5.162 TELLNEWS GETOLD * gives a core dump

Try

tellnews getold "*"

Need I say more? This gets me every time too :-)

5.17 TELLNEWS says remote node is unknown

Check the myname variable in DNEWS.CONF also check your DNS is working if its isn't then use the number in dnews.conf

On VMS if the symbol tellnews is not defined it will try and run telnet instead this can be confusing. You may have to log in again to get the symbol or if you forgot to put it in your sylogin: script then you will have to do so e.g.:

$ tellnews :== $dnews_top:tellnews

Return to the Index

5.18 TELLNEWS says "Cannot access tellnews.pass"

For some reason tellnews cannot find it's configuration files on NT this could be caused by a file protection problem or more likely if the disk drive letter was not specified in the configuration file DNEWS.CONF.

On VMS and UNIX you must be running the tellnews command from the usercode ROOT or NEWS or NEWS$SERVER.

Return to the Index

5.19 I see errors "NNTP Sorry I can't talk to you" in the log file?

Your news-host does not want to talk to you this is because there is a security file which gives a list of hosts which are allowed to connect the name of your machine must be in this list and your machine's name must be in the Domain Name Server tables as well as the Reverse Translation Domain Name server tables.

The access file may be called something like:

CNEWS /usr/local/lib/news/nntp_access

INND /usr/local/news/nnrp.access

DNEWS /usr/local/dnews/access.conf

DNEWS-NT /dnews/access.conf

Return to the Index

5.20 What does "Transfer permission denied" mean?

This can come from one of two directions DNEWS can give this error when a site tries to feed it new items.

If you are using a sucking feed then you should stop the site that is trying to send to you from sending as DNEWS will ask for any items it wants.

If you do want to allow incoming IHAVE messages then add an IHAVE record for this ip address in the ACCESS.CONF file and also remove the 'POSTING' flag from the NEWSFEEDS.CONF file for the ME site.

The last possibility is that this error is coming from your feeding site in which case you need to contact the system manager of that site and ask him to give you 'ihave' access.

Return to the Index

5.21 Posts not going upstream 437 Duplicate article?

The problem is DNEWS is echo'ing all your incoming news back to your news feed site this is clogging the outgoing feed. To fix this see the section above entitled "3.6 How do I set my EXCLUDE parameter correctly?"

Return to the Index


6.0 I still can't figure it out how can I get help?

You have two options the DNEWS discussion list has lots of expertise on it. So try that first.

If that fails or if you are a registered (not free) customer then as part of normal support NetWin provide email advice and help on setting up your system. In general try and give as much information as possible about your setup and what you want to do and ALWAYS include a FAX or phone number just incase your mail system is broken as well :-) e.g.

Hardware & OS: Windows NT - Intel (or NT-Alpha or Solaris 4.3 etc...)

Version of DNEWS: 2.0b

Type of feed: Sucking feed (or full ihave feed...)

FAX: (include country code)

And any relevant setup files usually one or more of: dnews.conf newsfeeds.conf access.conf

And maybe a few lines from dnews.log showing the problem

(Please don't email 3MB of log files to us :-)

And state your problem clearly:

(a bad example) My feed doesn't work why not?

(a good example) The IHAVE feed coming INTO my system from my provider is being refused he says it gives an error 'transfer permission denied' I have searched the log files but can't find an equivalent error at my end what should I do?

Return to the Index

6.1 DNEWS Mailing List

There are two mailing lists for DNEWS one is only used to make major announcements of new version or urgent patches. The other is for general discussion you may want to join both.

Return to the Index

6.2 To subscribe to the discussion list send to:

dnews-request@adam.com.au

A message containing one word:

subscribe

Then to post to the list send to:

dnews@adam.com.au

Return to the Index

6.3 To subscribe to the announcement list

Send a message to:

dnews-announce-request@netwin.co.nz

Containing the line:

subscribe dnews-announce

This is a moderated list it's only function is to provide timely information on new features releases bugs installation gotcha's etc to interested people. You will not get 20 messages a day from this list it will probably average 1-2 messages per month.

Return to the Index


File Formats

Active.dat

        group end start modflag cacheflag feed1last life range feed2last feed3last feed4last gid
group = The name of the news group
end = The last item in the group
start = The first item in the group
modflag = m=Moderated group
y=Normal group
d=Deleted
g=Gone
e=Deleted-moderated
cacheflag = n = not cached
c = cached
feed1-4last = The last item successfully sucked from our upstream site.
life = Number of days to continue caching this group for.
range = Number of items in the group
-1 means it will be calculated when
needed.
gid = The GID given to this news group
these are unique and MUST NEVER change.

Active.times

This file is used by news readers to find new news groups.
      group time
group = group name
time = time in seconds that the group was created.

Active.names

Don’t worry to much if you don’t have this file
almost no news readers
even use it.
      group<tab>description

Chapter 11. Full Text Searching - An Extension to Dnewsweb

Warning: This is a beta release of the Full Text Searching mechanism which was more than a little difficult to implement it is still experimental at this time! We will be keen to hear how it performs in the 'real world' your feedback is always valuable.

Warning!!! To build a really complete index will require free space of about 10-50% the size of your spool area it will also gobble up significant processor & memory resources. See the file fts.conf to control how

much of each item is actually indexed and limit the list of groups that you send to the indexing service.

You can 'index' a subset of groups possibly just your local news groups which are important to you this will require very little resources.
[return to index]

What to do:

1) In dnews.conf define the directory to use to build indexes it must be a disk with lots of free space :-)

fts_dir c:/dnews/fts (NT)

fts_dir /var/spool/dnews/fts (Unix)

2) Create the directory and move or copy fts.conf into it.

(NT)

mkdir c:/dnews/fts

copy /dnews/fts.conf c:/dnews/fts

del /dnews/fts.conf (so you don't mix it up with the real one)

(UNIX)

mkdir /var/spool/dnews/fts

copy fts.conf /var/spool/dnews/fts

3) Install an xmit service to build the indexes:

(NT)

addsvc -del dfts

addsvc -add dfts xmit.exe "dfts ftsfeed -fts -at *5 *"

(UNIX)

nohup xmit ftsfeed -fts -at "*5" "*" &

(put that in a startup file)

4) Add an ftsfeed to newsfeeds.conf to send items to the xmit service. Select a list of groups that seem sensible to index don't index the binary groups :-) (add this to the end of newsfeeds.conf)

site ftsfeed

groups !* comp.* rec.* local.*

5) Install dnewsweb and in dnewsweb.ini tell it that fts is enabled

search true

6) From Program Manager start the xmit fts service:

In Control Panel Services Start the DFTS Service

(UNIX) nohup xmit ftsfeed -fts -at "*5" "*" &

7) Wait for the indexes to start being created if you have a small system you might want to send your current items into the database we DO NOT RECOMMEND using this command it WILL lock up dnews for several minutes:

tellnews refeed ftsfeed (DO THIS AT YOUR OWN RISK)

8) Try the system out:

Notes:

1) New items will not appear immediately as DNEWS does not reload the database very often.

[return to index]


Getting News Into your server

This section describes the settings necessary to get news on to the dnews server in the simplest way possible - from a single provider using either of the two main methods used to transfer news items.

Setting up a basic sucking feed

This is the default configuration and Dnews comes setup to do this straight out of the box.

In dnews.conf set the nntp_feeder to the site that you are going to suck from set the myname variable to the name of the machine that you are running dnews on e.g.

	nntp_feeder pull-feed-out.internetmci.com *
myname news.mydomain.com

Setting up a basic IHAVE (traditional) news feed

In access.conf give your feed site access to send you ihave messages sometimes you will have to give several names or a wild card to allow all the news servers that may send you items. Also it is wise to add the ip numbers if possible then you can turn reverse DNS translation off if necessary.

	news*.mci.com:ihave:::*
223.232.11.*:ihave:::*

In dnews.conf comment out the nntp_feeder definition and add a line:

	# nntp_feeder ...
cachedall true

In newsfeeds.conf replace

	site $NNTP_FEEDER
		groups *
		builtin
		posting 
With 
	site my.feeder.site
		groups *
		builtin 
And correct the ME feed
the default one does not allow any binary groups in
if you really want a
full feed fix this by removing the !*binary* 

For an IHAVE feed from a DNEWS Server

You can modify your incommng IHAVE feed to specifically select the groups that you want sent by using a special program that you access through the WWW on the remote DNEWS Server. Please follow this link for more information.

For an IHAVE feed you must get an active file

You MUST get a copy of active.dat from your provider the file is simply called the 'active' file on INN or CNEWS you need to FTP the file and copy it to \dnews\spool\active.dat while the server is NOT running. We do not supply an active file with the installation kit because the list of valid news groups is different in every country in the world. The full list is a combination of world wide groups and local groups. So you should get an active file from your own provider if at all possible. If they will not give you one then there is a reasonable one on our ftp site that you can use.

If you have a Sucking feed then DO NOT do this DNEWS will fetch the list of groups itself.

> tellnews exit (to stop dnews
it shouldn't have been started yet)
ftp> ascii
ftp> cd /usr/local/news
ftp> get active
ftp> quit
(NT) > copy active c:\dnews\spool\active.dat
(NT) > net start dnewssvc
> tellnews resetall 	(Never use this command again
it clears the values in your active file)

If you ever need to add part of an active file to a running dnews system use the GROUP_MERGE command to add groups that you don't already have.

Adding a sucking feed for a subset of groups

To add a sucking feed there are two legs to consider sucking the articles in and sending the articles out. Here is an example of sucking some local groups bank.* from a second server news2.bank.com and sending any posts back to that server:

In dnews.conf
nntp_suck2 news2.bank.com bank.*
update_at2 20 *
In newsfeeds.conf
site me
groups *
site main.upstream.site
type builtin
groups *
!bank.*
posting
site news2.bank.com
type builtin
groups !*
bank.*
posting

Adding a sucking feed for all groups

Some people want to suck news from two different providers this will 'fill in the gaps' if items are missing from one of the servers. Here is an example of sucking all the groups from two servers.

In dnews.conf
nntp_suck2 news-out.byke.com *
update_at2 20 *
In newsfeeds.conf
site me
groups *
site main.upstream.site
type builtin
groups *
exclude second.upstream.site

There's no need to send back to the second site the items will arrive there via the usual news distribution.

Sucking in UUCP or Satellite articles

DNEWS can read UUCP bag files in standard rnews format these are often used by satellite systems. To read these files just specify in dnews.conf the directory and file names that dnews should scan. e.g. in dnews.conf

         suck_uucp d:\dnews\uucp\*.bag

The rnews file format is a collection of articles in a single file each one starts

#! rnews nnn
(News article 1)
#! rnews nnn
(News article 2)

Where 'nnn' is the length of each news article in bytes counting one character for each end of line. In general the file should be in 'unix' format that is there should be no 'cr' characters each line should end in a line feed character.

Adding an IHAVE feed

Adding an incoming IHAVE feed is very easy you simply add to the access.conf file an entry to let that server send you items.

	second.sending.site:ihave:::*

The only thing you need to consider is if you wish to route items from this server to your other providers if you don't want to route these items then you must set your exclude paramters carefully so that no messages are routed between the sites e.g.

	site main.upstream.site
type builtin
groups *
exclude second.sending.site
site second.sending.site
type builtin
groups *
exclude main.upstream.site

If you do wish to route news between these two servers then you should be using live links to handle the volume e.g.

	site main.upstream.site
type live
groups *
site second.sending.site
type live
groups *

Example of full feed from UUNET

-------dnews.conf--------- 
# nntp_feeder your.feeder.ohio.edu 
cachedall true
------newsfeeds.conf cut here ---------- 
site me 
groups * 
site news.uu.net 
type builtin
groups *
!local.* 
exclude uunet 
site YOUR.MAIL.GATEWAY 
smtp 
------access.conf cut here and change your domain and ip nnumbers ------ 
# Don't let the world connect :-) 
*:logoff:::*
# Do let your domain and ip range connect **** CHANGE THIS BIT *** 
*.your.domain:Read
Post:::* 
161.232.32.*:read
post:::*
# Do let your news feed from UUNET send you items 
news*.uu.net:ihave:::*
[return to index] 

Guide to USENET news and DNEWS

The following is a brief guide to explain how Usenet news works and a walk thru the the main configuration files that control DNEWS we suggest you read thru this in a linear fashion to gain a good basic understanding of News.

How UseNet News works?

The first question you should ask is how does news work how does a news message get from your computer to every other news server in the world and versa visa how does a message get from somewhere in Japan to your machine.

Many people imagine the news servers are connected in a tree like pattern the message would go up from your machine to the 'top' of the tree and then propagate down to all the other news servers.

This is entirely incorrect. Although simple in concept this would not work well in reality and because the internet is more of an anarchy than an organized system so the news topology is much more random.

The real topology of news servers   

Note that systems are more or less randomly connected together there is no ‘top’. This structure results in three conditions that a server can find itself in.

  1. Tails machines connected to only one server
  2. Feeders servers connected to 2 or more servers
  3. Loops servers which are part of a loop so the same message could arrive from two directions at once.

Imagine a message posted on “Goofy”. The message is stamped with a 'PATH' header which is added to as it passes thru every machine. If a host name is in the path then the server knows not to send it to that host again.

Here is what happens to the 'path' header as a messages travels from machine to machine:

On Goofy: Path: Goofy
On Mickey: Path: Mickey:Goofy
On Mars: Path: Mars:Mickey:Goofy
On Pluto: Path: Pluto:Mars:Mickey:Goofy

On Venus: Path: Venus:Mars:Mickey:Goofy
On Sun: Path: Sun:Venus:Mars:Mickey:Goofy

It may sound dumb but a news server will use this logic even to decide about sending a message straight back to the host it just got the message from. And becuase machines have multiple names it's possible for this logic to fail when this happens your news server will try and send every single message back up stream. (A waste of network bandwidth)

Above the message is about to be delivered to 'Earth' from two directions. Here even with the path header working correctly both machines will try and send to 'Earth' (because neither has ‘Earth’ in it’s path header). How does Earth know that it already has this message? Enter the history database...

Each message is gauranteed to contain a unique message ID. The news server stores these in a database and with every incoming message it first checks if it currently has that message ID before it stores the new message.

This database which could be 100MB’s has to be very fast DNEWS probably has the fastest history database of any news server.

Pseudo code for processing an incoming message

NNTP Protocol / commands

200 161.29.2.2 DNEWS Version 2.4d posting OK
help
mode reader
list
group rec.humor
article head body number|<message id>
ihave post
 

NNTP works with simple TEXT commands using telnet you can run NNTP by hand.. On some computers (NT 3.5) you need to type ^J after each command instead of just ‘enter’ There will be no ‘echo’ so you have to type carefully :-) here is a list of some of the NNTP commands you might use by hand.

helpGives a list of valid nntp commands for this server
mode readerTells it you are a news reader not a machine trying to send a feed in.
listGives a full list of news groups this could take 20 minutes!
group x.ySets the current group and shows you the number of articles and the first and last item in each group
article nnnShows you an entire article you can specify it by item number or by message
ihaveOffers the server a news message it responds with please send the message or no thankyou.
postOffers the server a NEW article.

NNTP example

telnet news.server.name 119
200 161.29.2.2 DNEWS Version 2.4d posting OK
mode reader
200 DNEWS Posting OK.
group rec.humor
211 299 101760 104058 rec.humor selected
article 104058

NNTP Protocol & extensions

Here are some important NNTP commands that you should understand:

ihave <message-ID>
check takethis
post
xover
xhdr

When a news server wants to send a feed at your news server it would traditionally use the 'ihave' NNTP command. Here is how the traditional IHAVE protocol works:

Send: ihave <messageid>
Waitfor: Wanted or Not Wanted
Send Message (if wanted)

This protocol doesn’t allow streaming as it is necessary to wait until the server responds before sending the message or skipping.

The new NNTP commands 'check' and 'takethis' are basically the same as ihave but the operation is split in half so that a feed can send 100 check’s and then send ‘takethis’ followed by the articles for the ones that the server wanted.

The 'XOVER' command was added to allow a directory of articles in one group to be shown without downloading the header information from every article in the group. Example:
xover 1200-1400
The server responds with 200 lines each line contains the interesting header fields for one message separated by tabs.

A raw NNTP article

From: bn946@FreeNet.Carleton.CA (Les Griswold)
Newsgroups: soc.couples.intercultural alt.politics.white-power alt.fan.oj-simpson
Subject: Re: Beaver's pride
Date: 23 Apr 1996 09:06:26 GMT
Message-ID: <4li6ii$fj8@freenet-news.carleton.ca>
NNTP-Posting-Host: freenet3.carleton.ca

This is the message here
which I’ve actually just made up.

Problems

Traditional news servers have the following problems:

  • 20 000 News Groups
  • 128Kbps 24 hours / day
  • 146000 articles / day
  • INN installation
  • INN management
  • Disk Use
  • Memory Use

DNEWS Sucking

  • Only groups which users read are sucked
  • All groups are available for the user to choose from without having to ask
  • Standard NNTP ‘reader’ protocol is used so a feed doesn’t have to be setup
  • Low impact history lookups avoided newnews not used

DNEWS sucking results

1-10% of disk space used
1-10% of network bandwidth used
Messages can be stored much longer
Nightly expire run down from 24 hours to 5 minutes
Much easier to setup at both ends.

DNEWS Configuration Files an outline

Expire.conf - Intelligent Expire settings

Expire example

upto 100% groups * keep 1 default 5 purge 30 maxitems 2000 minitems 10
upto 90% groups * keep 1 default 15 purge 30 maxitems 3000 minitems 100
upto 50% groups * keep 4 default 30 purge 30
upto 10% groups * maxitems 4000 minitems 100
upto 100% groups local.* minitems 10000

For each news group the whole file is searched to find the LAST rule that applies for the current disk usage e.g. if we are currently using 49% of available disk space and this group is called rec.humor then the

upto 50% groups * keep 4 default 30 purge 30

 

keepHow many days to keep an item even if it has an expiration date in the past
defaultHow many days to keep an item with no expiration date
purgeMax days to keep an item even if it has an expiration date
minitemsIf less than this many items in a group then don’t delete any items despite what the other rules say
maxitemsIf more than this many items then delete oldest ones despite what other rules say.

 

Access.conf - Powerful and tricky!

The access.conf file is very powerful it can be used to control access to your server in many different ways and as a result it is also very tricky to use. The main source of confusion comes because of two features first the file is passed twice once when a user connects and a second time when/if they supply a password. The second tricky bit is that the 'last' matching rule in the file is used. So the order of rules is very significant.

This file can do all of the following:

Access example

*:logoff:::*
*netwin.co.nz:read post:chrisp:passwd:*
*netwin.co.nz:read post:::* !*sex*
161.29.1.*:read post:::* !*sex*
news*.mci.net:ihave:::*
mypc.netwin.co.nz:read post:::*

 

*:logoff:::*Immediately closes a connection from anyone outside our domain
*netwin.co.nz:read post:chrisp:passwd:*Allows me to read and post to any news group (including the naughty ones) as long as I configure my news reader with the username: chrisp and password: passwd
*netwin.co.nz:read post::* !*sex*Allows users to read and post to any newsgroup except ones with the word ‘sex’ in the name.
news*.mci.net:ihave:::*Allows an ihave feed from an MCI machine e.g. news12.mci.net
mypc.netwin.co.nz:read post:::*Allows me to read all groups even the naughty ones from my own PC as long as it has a reverse DNS entry.

Note: netscape cannot be configured to give a user/password from the same domain that allows connections without user/passwords. This is a netscape bug caused by their having mapped the web security onto news and as web has the same limitation they have caused the same limitation to become a feature of news.

Note: reverse DNS translation is often not properly setup don’t rely on it always give ip numbers as well as names. You can give wildcard numbers like 122.33.44.* to allow everyone from a given subnet. Also consider using the 'nodns true' setting which stops DNEWS even trying to perform the reverse name translation this can improve performance.

Control.conf - How newsgroups get created

## <message>:<from>:<newsgroups>:<action>
all:*:*:mail
newgroup:*:*:mail
newgroup:tale@*.uu.net:comp.* news.* rec.* sci.*:doit
newgroup:kre@*mu*au:aus.* melb.*:doit
rmgroup:*:*:mail
rmgroup:tale@*.uu.net:comp.* news.* rec.* sci.*:doit
rmgroup:kre@*mu*au:aus.* melb.*:doit
 

Who really runs NEWS? Who creates news groups once a new group is approved how is it created on 10 000 news servers all over the world.

The control.conf file is the key to this. It specifies who else in the world is allowed to create newsgroups on your news server. Control messages are just like normal news messages except they have a special header:

Control: newgroup rec.humor.really.funny

When DNEWS receives a control message it is checked against the control.conf file. The last matching rule is used to decide what to do. The possible actions include:

 

logLog the comand to a file control_COMMAND.log you would then edit this file to remove any commands you do not wish to have actioned and then rename it to '.cmd' and run it as a batch process
mail EMail the message to the DNEWS manager
dropJust throw away the message
doit Automatically perform the action

DNEWS Manager

DNEWS manager allows you to perform two major tasks it lets you configure the above configuration files us a nice windows GUI and it lets you send tellnews commands to the running news server.

TELLNEWS commands

Tellnews commands are used to control and query a running dnews news server they are actually handled by dnews itself the tellnews or dnews manager process simply creates an nntp connection to the server and sends thru the tellnews command and then listens for dnews to send back a response. Tellnews commands are used to:

Chapter 13 The Gui's - Windows NT Windows 95 Macintosh

The Windows versions of dnews ship with the Dnewsman interface the Macintosh version has a similar interface called the Tellnews Control. These are graphical interfaces that allow the user to control the dnews server including starting and stopping the dnews service

The DNEWS Manager allows easy setup and management of the news server. The following chapter describes what can be done from each dialogue.

The front dialogue is divided into two main areas on the left options for configuring DNEWS and on the right options for controlling the running system and sending interactive commands to it.


Setup DNEWS Server - Configuration Options

Main Options

This dialogue covers the major settings in dnews.conf that need to be configured.

My IP Name

This is the ip name or number of your news server. It must be a working name or number which you could use to connect to your PC via FTP or TELNET. If you have multiple IP numbers then use the local ethernet IP number. If you use a name that doesn't work the status and other COMMANDS won't work. (and may lock up for extended periods when you try them)

Suck News From

This is the ip name or number of the server you are going to fetch news from. If your name server is flaky or not always available (e.g. with RAS) then using the IP number can avoid lock ups. You need simple NEWS READER access to this machine.

If you are getting a full IHAVE feed from a site (e.g. they are pushing the news at you) then this item should be left blank.

Disk Space (MB)

DNEWS will try and keep it's disk usage below this setting. A value of 300 is good for about 200 users but it is dependent on their reading habits.

Manager Email

DNEWS will send you a daily report of the expire process. This should be a valid internet email address e.g. joe@my.domain

Confirm Postings

DNEWS can send your users confirmation messages when the local postings are sent up stream this setting lets you tell DNEWS who your users are (as it doesn't want to send confirms to every person on the internet who sends a message 'through' your machine. You must include a wild card (* character) in this. e.g. *this.domain. Note: *.this.domain would not match joe@this.domain whereas *this.domain will. (the extra full stop causes the problem)

If you run two domains you can give both domains separated by a comma e.g.

*this.domain
*other.domain

Warn Info Debug Flush

The default logging level is 'info' If you experience crashes NetWin support staff will ask you to set the debugging level to FLUSH and then send the log file next time a crash occurs. The debugging level will slow down the server.

Try to translate IP numbers to Names

When a user connects DNEWS can try and translate the ip number (e.g. 161.203.12.5) into an ip name (e.g. pc1.my.domain) But this translation could lock up your server if your DNS is faulty (or if it can't be contacted for some reason) So you may want to stop DNEWS from doing this translation. However if you do this you must put ip numbers into access.conf (the user access file) because it will not be able to check the ip names.

Advanced…

This dialogue lets you modify advanced settings from dnews.conf
Advanced Settings Dialogue Box

Cached Life

The number of days that a group should remain 'cached' after it has been read by a user. When a group is 'cached' items will be fetched on the off chance someone will want to read them. This setting has nothing to do with how long items are stored for after they have been fetched that is set in the configuration file expire.conf

Expire At

The time in hours and minutes that DNEWS should run it's automatic expire job. Either value can be a wild card or a list so if the minutes were set to 15 45 then the expire would run at quarter to and quarter past the hour. If the hours were set to 3 20 then an expire job would run at quarter past 3 and quarter to 4 and quarter past 8p.m. and quarter to 9:00p.m.

Update At

How often DNEWS should fetch new articles this is runs every 10 minutes by default. It would not hurt to decrease this to once per hour. e.g. minutes=15 hours=*

Rotate Log

The log files tend to grow very rapidly in order to stop them getting too big they can be automatically rotated (deleted) every 'n' lines.

Config Files

This setting is where the config files are don't play with this!

Spool Files

This setting is where the spool directory is you may need to move the spool area to a larger disk if you want to have heaps of news. To do this:

stop the server

Move the spool tree using file manager

Change this setting using DNEWS Manager

Start the server

Cross your fingers :-)

Sucks

This dialogue is used to set up the site or sites that Dnews will suck from.


Multiple sucking feeds can be set by supplying a wild card of news groups to fetch for each sucking feed. The largest set should be the first in the list.

E.g.
suck1  main.news.site *
suck2  local.news.site secret.*

Feeds Out

This dialogue lets you modify the settings of the selected feed.

Users and Feeds In

This dialogue lets you control access to your server.

Each line in the access.conf file is checked when a connection is made both ip numbers and names (if reverse dns translation is enabled) are matched to the first column. If a match occurs then the READ POST IHAVE and GROUP settings are then assigned to that user.

After connecting the user can then supply a username and password when this happens the file is checked again and the read post ihave and group settings are modified for that connection.

Note also that every line in the file is matched and the last match is used so you have to put general settings at the top and specific user settings near the end.

Here is an example of allowing a user to connect and then give user/passwords

*:logoff:::*
*:read post:joe:joepassword:*
*.my.domain:read post:::*

This will prevent the rest of the world from even connecting while letting most people read and post mesages Joe will be able to connect from any domain as long as he uses his username and password.

If you use the Netscape browser then there are limitations on how you can use this file this is because netscape doesn't allow you to enter the username and password unless access is refused by the server. This isn't generally a problem but it can cause a problem if you want to have people from one domain connecting with and without passwords this will not be possible via netscape but will work with all other browsers.

Setup Wizard

This dialogue will step you through the basic DNEWS setup it is intended to get the system running when the person answering the questions doesn't really know what they are doing :-)

It should only be run ONCE when you first install DNEWS as it may change settings which you have since modified.

1. Type the ip name or ip number of the machine that you are running dnews on in the available box and then click on the OK button.

2. If you want to set up a sucking feed click on the check box then the OK button if you will be taking an Ihave feed ensure that the box is not checked then click on the OK button.

2a. If you checked the box to suck news then you'll need to set where news is to be sucked from type in the ip name or the ip number of the site that is providing you with a sucking feed.

3. In this dialogue box you need to enter the name of your mail gateway this may or may not be the machine which you are running dnews on. If this is the same machine you need to use the mail alias which will be different from the name that you set for the dnews machine in box 1. Type in the name and click the OK button.

4. Dnews produces a daily report supplying a number of statistics about what it has done over the preceding 24 hours type the address of the person to mail these reports to into this dialogue box and click on OK.

5. Decide how much space to give dnews to store news on for an Ihave feed with binaries groups 4000 to 10000Mb are needed without the binaries groups 2000 to 4000Mb will be sufficient. With a sucking feed 200Mb is about the minimum required to accept a feed. If you are using Dnews as an intranet server and will have only local groups then as little as 30Mb may be enough.

6. When someone posts to a group dnews can send a confirmation that their post has been successful type in the address range that dnews should send confirmations to. E.g. you@machine.your.domain will send confirmations to you only *@machine.your.domain will send confirmations to anyone who posts from an account on the machine *your.domain will send email confirmations to anyone who posts from that domain.

Type in the ip numbers of the people you want to allow to read news and click on OK.

Type in the ip names of the people you want to allow to read news and click on OK.


Write Changes

This button will write any changes you have made to your configuration files. If you don't press this button the changes will be lost.

WARNING: Generally you should make changes then press "Write Changes" then press "Do Reload" to force the DNEWS server to reload it's configuration files.

Manage Server - Command Options

Stop Service

This button stops the DNEWS service (it will not work on Windows 95 or from a remote PC using a mapped drive in those cases you should use the Commands dialogue and send an EXIT command.

Start Service

This button attempts to start the DNEWS service it will not work from a remote workstation using a mapped drive or on Netware or on Windows 95).

For Windows 95 you must start the service by double clicking on the dnews.exe icon.

For Netware you must load the NLM from the console (or with rconsole)

View Log

This button shows the last 100 odd lines of the dnews.log file. After starting the server you can use this button to find out when the server has actually finished it's initialization. Also in the event of any unexplained behavior you should check the log file.


Status

This buttons sends a 'status' command to the server and shows you the output if it locks up doing this then you probably have a problem with your reverse name translation which you can disable in the main setup dialogue.

Commands

This dialogue lets you send any of the builtin TELLNEWS commands to the server. If the command has parameters you will be prompted for them. You can use this to add new local news groups or perform other strange and wonderful management tasks.

Do Reload

This button send DNEWS a reload command this will make the server re read all it's configuration files You need to press "Write Changes" before pressing the reload button.

WARNING: This button DOES NOT reload the configuration files into DNEWS Manager if you change the files outside of DNEWS Manager then exit it and restart it to pickup your changes.

Config Files

This dialogue lets you tell DNEWS Manager where to find the DNEWS configuration files.

Use this if you are running DNEWS Manager from networked PC first SHARE or MAP a drive to the disk where DNEWS.CONF and the dnews configuration files are stored and then run DNEWS Manager and tell it which drive and directory to look in.

FEED Modify

This dialogue lets you modify the settings of the selected feed

Note the first FEED must be the ME feed. This is a dummy feed which is actually used to check if incoming items match the list of news groups that you want.

If you change the ME feed you may have to send the TELLNEWS COMMANDS:

undelete *

matchfeed

Don't do this without first understanding what they mean :-) (see the manual)

Access Modify

This dialogue lets you modify the settings of the selected access line.

Read access lets users read news items

Post access lets users post news items

Ihave access lets an IHAVE feed come into your machine

Registering DNEWS.

To register to DNEWS click on the REGISTER button in DNEWSMAN and fill in the questions.


Alternatively you can use the TELLNEWS REGISTER command from a COMMAND SHELL. We do require payment before issuing a license key you can make payment by giving credit card information with your registration. If you wish you can fax credit card information to us on +64 9 630 0689. Other forms of payment are also acceptable Bank transfer Cheque Purchase Order etc.

After answering all the questions you can then start up your favourite email system create a new message and then press SHIFT+INS to paste the registration form into the message.

Send the message to your local DNEWS reseller or netwin@netwinsite.com in 24-48 hours your will receive a license key to fully enable dnews.

Entering your registration Key (when you get it via Email)

Enter the key with the tellnews key command or from the COMMANDS button on DNEWS Manager.

e.g.

         tellnews key a1b2c3d4e5

[return to index]


GUPCGI

groups update program common gateway interface - for use via the WWW

Information:

This clever little program will allow you to modify the groups that your news server receives from other news servers through the WWW. It is individually password protected for each site that you send news to so that access to it can be given to the other server administrators so they can modify what is sent to their news server. This is incredibly useful for a server administrator as they do not have to contact somebody from their upstream news source to tell them (nicely) to modify what is sent to them by an ihave feed.

Please Note this!

The information gets saved but it is not updated on the news server until DNEWS reloads the newsfeeds.conf file to make this happen automatically schedule a cron job (or use the "at" comand on NT) to issue a "tellnews reload" every few hours.

Installation: - 3 easy steps (only one of them is recursive)

  1. Copy the two files gupcgi.exe and setfeed.ini into you web server cgi-bin directory or where ever the CGI programs get run.
  2. Add into the setfeeds.ini the location of your newsfeeds.conf file "newsfeeds newsfeeds_location" where newsfeeds_location is a string identifying the file location of newsfeeds.conf i.e. "newsfeeds n:\dnews\newsfeeds.conf".
  3. Modify the setfeeds.ini file for the groups that you wish to be able to access via the web. The format specifications are in the setfeed.ini file. Basically a line with "site site_name password" is the format. You can specify as many sites as you like.

You and your fellow server administrators are now ready to run your GUPCGI . You may wish to add the address of it to one of your links pages. Note that DNEWS MANAGER should not be left running if gupcgi is being used.

Note for UNIX systems

On UNIX systems you will have to modify the protection on the newsfeeds.conf file typically this is done by giving the web user (usually 'nobody') access to write to the newsfeeds file e.g.

chmod 0770 /usr/local/dnews /usr/local/dnews/newsfeeds.conf
And then add 'nobody' to the 'news' group in /etc/group
[return to index] 

Installing Dnews


VMS_Installation

1) FTP the files from ftp://ftp.netwinsite.com/pub/netwinsite/dnews/vms/alpha (or /vax)

2) @extract Extracts the required files from the saveset

6) $ edit sys$manager:systartup_vms.com Add a call to dnews_start.com

7) $ edit sys$sylogin: Add the symbol see dnews_login.com

8) $ edit dnews_top:dnews.conf

Modify the file DNEWS.CONF to describe your system by defining the following symbols

SYMBOLEXAMPLE
managersystem (email address or user@site.com)
nntp_feedernews.feeder.site
mynamenews.this.place
timezonenzst (new zealand standard time)
diskspace100 (space available in megabytes)
mailsmtp (the foreign email protocol name)
confirm*.this.place Enable email confirmation when item is passed up to a feeder site.

9) If your news feeder stamps news items with a name other than the ip name you specified above then you will need to add an exclude field to the file newsfeeds.conf to prevent your system echoing all news back to the feeder!!! (check the path header of an item to see what it is doing)

10) $ edit access.conf Grant an ip range access to your users. e.g.

*.this.place:read
post:::* (by name)
161.23.4.*:read
post:::* (by number)

11) $ @dnews_start Start the system

$ @dnews_login (to define the symbol tellnews)

The system will now try and get a list of news groups from your feeder site while this is happenning it will not respond but you can check it's progress by:

$ type dnews.log

When it has finished (give it 10-20 minutes) you can then try:

$ tellnews status

  1. Go to a PC and try out your news reader software pointed at the new system.

[return to index]


UNIX Installation

If you are upgrading UNIX from version 2 to version 4 (with multiple drives connected using symbolic links) then you will need to follow this link for up grading instructionsUnix Upgrade Notes.

Upgrade process this is automatic and should take approx 1 hour per gig. WARNING if your spool area is split over multiple drives using symbolic links (unix only) then you must follow the instructions below on upgrading unix multiple drive systems.

1) FTP the files from ftp.std.com ftp/vendors/netwin/dnews...

dnewsVER_MACHINE.tar.Z (binary)

(e.g. dnews30a_linux.tar.Z )

2) Extract to a temporary directory in your own account

uncompress dnews30a_linux.tar

tar -xvf dnews30a_linux.tar

3) EDIT dnews.conf

Modify the file DNEWS.CONF to describe your system by defining the following symbols

SYMBOLEXAMPLE
managersystem (email address or user@site.com)
nntp_feedernews.feeder.site
mynamenews.this.place
dfdf -k (path and options for df command to get posix standard output df -Pk somtimes works)
diskspace100 (space available in megabytes)
sendmailsendmail (The path and flags for sendmail or equiv.)
confirm*.this.place Enable email confirmation when item is passed up to a feeder site.
xover_dirDirectory to store xover index files (allow 10% of spool area) defaults to WORKAREA directory

4) Edit your system startup script the name will vary:

       On LINUX /etc/rc.d/rc.local 

Add a line:

       /usr/local/dnews/dnews_start.sh 

5) If your news feeder stamps news items with a name other than the ip name you specified above then you will need to add an exclude field to the file newsfeeds.conf to prevent your system echoing all news back to the feeder!!! (check the path header of an item to see what it is doing)

6) Edit access.conf Grant an ip range access to your users. by name or number e.g.

    *.this.place:read
post:::*
    161.23.4.*:read
post:::* 

7) If you have changed any of the default directories then edit all the .sh scripts to reflect the changes.

8) Run the installation script

    su root (You need priv to install the files start dnews)
    ./install.sh 

9) Start the system

    cd /usr/local/dnews
./dnews_start.sh 

The system will now try and get a list of news groups from your feeder site while this is happenning it will not respond but you can check it's progress by:

    more dnews.log 

When it has finished (give it 10-20 minutes) you can then try:

    tellnews status

10) Go to a PC and try out your news reader software pointed at the new system.

LINUX slackware:

Comes with a dummy nntp daemon installed you must remove this so dnews can bind to the socket. (so it won't even start up)

edit /etc/inetd.conf (comment out the NNTP line) 
kill -HUP <pid of inetd process> 

BSDI Unixware Sunos

Some brands of unix (BSDI Unixware Sunos) have process limits the two that will hurt dnews are memoryuse/datasize and openfiles. The following commands can remove these limits you will need to do this for root and news users.

     limit memoryuse unlimited
     limit openfiles 256
     limit datasize unlimited 
     (Bash Sunos increase file handles) ulimit -n 256

Here is an exmple of dnews_start.sh for BSDI

#!/bin/sh
cd /usr/local/dnews
ls -ald /usr/local/dnews
chown news /usr/local/dnews/*
chown news /usr/local/dnews
rm nohup.old
mv nohup.out nohup.old
(The next two lines are for BSDI only)
sysctl -w vm.maxdsize=120000000
sysctl -w vm.maxssize=90000000
limit datasize unlimited
limit openfiles 256
limit memoryuse unlimited
nohup ./dnews_start &
  To test these limits are really removed use the commands:
     tellnews test_mem
     tellnews test_files 

You must stop and restart dnews after running these tests. You should see about 200 free file handles and about 30MB or more of memory that can be allocated if you see less then you haven't removed the limits.

Unix Upgrade Notes (with multiple drives connected using symbolic links)

In dnews.conf use the setting:

spool_free true

And define each of your drives as one of the linked drives. e.g. if you currently have a second drive mounted as /var/spool/dnews/alt/binaries then your dnews.conf should look like this:

spool /var/spool/dnews
spool2 /var/spool/dnews/alt/binaries
diskspace 2000
diskspace2 1000

After the conversion has finished it is safe to move the directories but you cannot do this before the conversion because dnews must be able to find the 'old' files where they 'used to be'.

[return to index]


Windows NT Installation

1) FTP the files from ftp://ftp.netwinsite.com/pub/netwinsite/dnews/dnews30a.exe

2) Run the self extracting archive and installation procedure

      dnews30a.exe

3) Run DNEWS Manager to modify the setup further and to start the service

4) Point your news reader software at your NT and try out your new fast news system.

5) Join the mailing lists for advice an update information and don't forget to register :-)

Open DOS Window

then type TELLNEWS REGISTER

6) Instructions on setting up dnews to take a full IHAVE feed.


Windows NT Upgrading

Shutdown the server and exit from DNEWS Manager

Run the self extracting archive

dnews40j.exe (or whatever version)

When installation is complete start DNEWS Manager from the DNEWS Program Group. And click on 'Start Service'

If upgrading from 2.x you may want to define XOVER_DIR in dnews.conf to specify the directory to store xover index files (allow 10% of spool area) defaults to WORKAREA directory.

[return to index]

Notes for Windows 95 Installation

Follow the instructions for Windows NT and mostly it will work except for:

1) Services don't exist so you have to run dnews from a DOS prompt. You can put this into your STARTUP group to make it start automatically like an NT service.

2) NetWin do not recommend the use of Windows 95 as a full news server for large numbers of users. Use NT if possible.

[return to index]


NETWARE 4.x Installation

WARNING: MEMORY

DNEWS expects virtual memory as netware has none you have to make sure your system has sufficient REAL memory for a sucking feed you will need 8MB free for a full IHAVE feed you will need 16MB-60MB of free memory (these are estimates)

WARNING: GROUPS

This release for NETWARE is limited to 10 000 groups this is heaps if you take care with the 'ME' setting in newsfeeds.conf

WARNING: BETA RELEASE

This is a beta release NetWin do not recommend that you install this on a mission critical server. Please report any problems you find.

attach myserver

File Run dnewsnw23f.exe (or whatever the latest version is)

Answer the configuration questions.

In your autoexec.ncf script add the following lines.

load dnews

Load the server by hand.

load dnews

Go to a PC and try out your news reader software pointed at the new system. On windows 95 you can run the DNEWSMAN configuration utility to setup or modify your configuration.

Use the command:

load tellnews status To see what it's doing

load tellnews register To see what it's doing

load tellnews exit To shutdown dnews nicely

load tellnews help For other commands.

NEVER shutdown the service with the 'unload' command. Always use load tellnews exit instead.

[return to index]


OS/2 Installation

1) FTP the files from netwinsite.com

ftp://netwinsite.com/dnews/dnews3.XX_os2.zip

2) Unzip the files

pkunzip c:\dtemp\dnews2.XX_os2.zip

3) Run install.cmd to rename files and copy them to the destination (typically c:\dnews) you can edit this script to define the destination first:

e install.cmd 
     set config=d:\dnews 
Then run the installation: 
     install 

4) Edit dnews.conf

e dnews.conf

Modify the file DNEWS.CONF to describe your system by defining the following symbols

SYMBOL EXAMPLE
manager youremail@your.site
nntp_feeder news.upstream.site
myname this.machine.name
diskspace 100 Space available in megabytes.
Confirm *.this.place Enable email confirmation when item is passed up to a feeder site.

5) Add dnews to the startup folder:

6) If your news feeder stamps news items with a name other than the ip name you specified above then you will need to add an exclude parameter to the file newsfeeds.conf to prevent your system echoing all news back to the feeder!!! (check the path header of an item to see what it is doing)

7) Edit access.conf Grant an ip range access to your users.

e access.conf
*.this.place:read
post:::* (by name)
161.23.4.*:read
post:::* (or by number)

8) Start the server by double clicking on the dnews icon in the startup window.

Check the log file and status to see if it's running: 
   > cd \dnews
> dtail dnews.log
> tellnews status 
Now try reading news
it may take a minute or two to fetch the list of groups from your server.

9) Go to a PC and try out your news reader software pointed at the new system.

10) If you have a full IHAVE feed coming in then you must read the instructions in the manual for taking a FULL IHAVE FEED.

NOTE: OS/2 Version 3.0 has a bug in it's tcpip stack the result of this is that when sucking dnews will sit waiting for the end of a news article that never arrives (because the tcpip stack has thrown away the data). Eventually the connection times out and dnews continues fairly normally. We found upgrading to OS/2 Version 4.0 fixed this problem.

[return to index]


Macintosh Installation

After running the self extracting archive open the file install.txt for instructions.

. [return to index]


Calculating memory use

This is a rough guide to calculating how much memory dnews will use. This is VIRTUAL memory you dont' need this much physical memory (but it will go faster if it does :-)

   (Concurrent connections) * 200K
(Articles in history file - see expire processed) * 12
(Spool Gig's) * 1MB
(Max article size accepted) * 2
(Number of news groups) * 100
head_cache (default 200) * 3k
xover_cache (default 3000) * 300
Program static data 4mb

So lets take a smallish system with 20 000 news groups 10 concurrent connections 200 000 lines of history data 1 Gig of spool 1MB articles.

   10 * 200K = 2MB
200
000 * 12 = 2.4MB
1Gig = 1MB
1MB Artilces = 2MB
20
000 News groups
= 2MB
head_cache (default 200) * 3k = 600K
xover_cache (default 3000) * 300 = 900K
Program static data = 4mb
Total memory use = 14.9MB 

NOTE you DO NOT NEED THIS MUCH RAM dnews is designed to deal with multiple full feeds and 100's of users if you are running a smaller system you don't need all this as REAL ram that's what virutal memory is for it just slows down a little.

Typically on a small system where you wish to reduce the memory use you need to look at:

You can drastically reduce memory use with these settings:

(in expire.conf) remember 4
(in dnews.conf) item_max 200000 (stops large binary items!!!)
(in newsfeeds.conf change your ME feed to only allow groups
that really exist in your country-area)
    site ME
groups !*
rec.*
comp.*
alt.*
news.*
soc.*
bit.*
bionet.*
sci.*
    tellnews matchfeed
tellnews purgegroups 

(restart dnews to free up the memory)

As a rule of thumb to avoid calculations with unix and NT create a 100MB swap/page file as a first step. If you are running a big system then 200mb would be a wise choice.

[return to index]

Adding PGP authentication to DNEWS 3.1

  1. Install PGP software on your computer
  2. Add the public keys of the public news administrators
  3. In DNEWS.CONF specify the path to the PGP program
  4. In control.conf add " pgp" to the 'action' field

Install PGP software on your computer

For legal reasons we cannot supply PGP directly as this would break certain archaic U.S. laws. However we can tell you how to get it: (apparently non U.S. citizens cannot read english :-)

For Windows NT :- look for a version for '32-bit DOS...' you don't need the fancy windows front ends (which do not work without also getting the DOS application)

For Unix:- get the source and build it typically with a command like:

   make linux

We would have liked to give you refences to the actual files but then someone might have accidentally downloaded the wrong one and then ended up being arrested for exporting munitions...and we wouldn't want that happenning would we :-)

Add the public keys of the public news administrators

Transfer the file http://ftp.sunet.se/pub/news/misc/pgpcontrol/PGPKEYS and then add these public keys to your key ring with the command:

    pgp -ka pgpkeys

In DNEWS.CONF specify the path to the PGP program

This step is only necessary if you have not put 'pgp' into your path.

    pgp c:\app\pgp\pgp.exe

In control.conf add " pgp" to the 'action' field

newgroup:tale@*.uu.net:comp.*
misc.*
news.*
rec.*
sci.*
soc.*
talk.*:doit
pgp
rmgroup:tale@*.uu.net:comp.*
misc.*
news.*
rec.*
sci.*
soc.*
talk.*:doit
pgp

Note: currently PGP is only implemented on theWindows NT and UNIX version of DNEWS.

Related useful links

Sending news out of your server

Sending LOCAL posts upstream using a POSTING feed

With the posting mechanism DNEWS pretends to be a news reader and sends items into the upstream news server as if it was itself a news reader. The only tricky bit is setting the exclude parameter correctly. In newsfeeds.conf

        site me
groups *
site $nntp_feeder
type builtin
posting
groups *
exclude news.pbi.net

The variable $nntp_feeder is replaced by the name you have used in dnews.conf as your nntp_feeder. A posting feed is not entirely legitimate although perfectly workable you should use an IHAVE feed to send articles upstream if your provider will allow it. The main advantage of the posting feed is that it works when you only have normal news reader access to your provider.

Note: NEVER remove the 'me' feed from the top of the newsfeeds.conf file it MUST be the first feed in the file.

Sending LOCAL posts upstream using an IHAVE feed

The IHAVE feed is the normal way to send articles upstream to convert a posting feed to an ihave feed simply remove the 'posting' keyword from the entry in your newsfeeds.conf e.g.

        site $nntp_feeder
type builtin
groups *
exclude news.pbi.net

The variable $nntp_feeder is replaced by the name you have used in dnews.conf as your nntp_feeder. Again you MUST get your exclude parameter correct.

Note: NEVER remove the 'me' feed from the top of the newsfeeds.conf file it MUST be the first feed in the file.

Sending a full feed onto someone else - Live Links

DNEWS 2.7 and later includes 'Live Links' this is where DNEWS holds open a channel to the site you want to send to and as items arrive they are immediately sent on to the down stream site. This is significantly more efficient than other systems as the item does not need to be read off disk for every feed it is sent to. In this way a server can support dozens of outgoing feeds with almost no impact.

The DNEWS implementation of Live Links is extremely advanced it includes the following features:

Here is a simple live link example (in newsfeeds.conf):

        site downstream.site.name
type live
groups *

Here is an example of a more complex live link using all the options.

        site downstream.site.name
type live
live_nchan 3
maxsize 100000
maxcross 5
groups *

 

Routing news between other news servers

If you want to route news articles between your system and other news servers then it is important to use the 'live' links rather than the 'builtin ihave or posting' feeds. A live links feed can process hundreds of thousands of articles every day. A builtin feed is only intended to process a few thousand every day.

If you have a local incoming ihave feed (from a local bulletin board for example) which you wish to send out thru a posting feed then you must use this setting in dnews.conf.

        post_route true

Shotgun posting

Some providers are not very good at getting posts out to the rest of the world or even if they do get out it takes many hours or even days to reach other major news systems. Shotgun posting is another innovation unique to DNEWS basically it is where you send local posts out to several distant servers to make sure it gets around the world fast. To get posting access to a server somewhere else you might offer posting access to your system in return.

        site main.feeding.site
type builtin
posting
groups *
exclude main.feeding.site
site distant.news.server
type builtin
posting
groups *
exclude main.feeding.site

Note the exclude parameter to prevent your system from routing items across to the distant site which is not your intention.

Writing uucp bag files

You can write rnews uucp bag files to a directory with a feed like this.

        site uucp1
groups *
type uucp
uucp_dir c:\uucp_bbs
uucp_size 4000000
uucp_n 500

Setting your exclude parameter correctly

In newsfeeds.conf you must set your exclude parameter to stop DNEWS from sending incoming messages back to your feed site. Lets assume your feed site is called fastfeed.world.net. You need to find out what it stamps onto news articles to do this read a news article (and show all headers so you can see the PATH header) e.g.

Newsgroups: alt.winsock.trumpet
Subject: Sportster Si does not seem to work with TRUMPET
Date: Thu
26 Oct 1995 22:44:08
Organization: Personal
Lines: 8
Message-ID: <moleski.2.0016BD02@sasknet.sk.ca>
NNTP-Posting-Host: eagle28.sasknet.sk.ca
X-Newsreader: Trumpet for Windows [Version 1.0 Rev A]
Path: fast1.world.net!world.net!news.icix.net!esol.intermedia.com

Look thru the path left to right and find the first or most generic name for your feed site in this case 'world.net' looks like the best name to choose.

So here is the entry you should put in newsfeeds.conf you may need the builtin and posting settings as well but that is up to you.

site fastfeed.world.net
groups *
exclude world.net
builtin

You may need to delete your .feed and .sending files from the spool directory to clear out the backlog of articles queued.

 

Chapter 12. Support Mailing lists etc.

Netwin provide email support for dnews generally this means that there will be an answer to any requests for assistance within 24hrs usually the response time is less than this. The address for email support is

netwin@netwinsite.com

There are two mailing lists the Discussion list runs at about 10 messages per day the Announcements list is very very low traffic.

Announcements:

 

You can join the dnews mailing list by sending a message to:

dnews-announce-request@netwin.co.nz

Containing one line:

join dnews-announce

This is a moderated mailing list it's only function is to announce new versions of dnews and to disseminate up to date information on installation problems bugs new features etc to people who manage news systems.

 

You will not be flooded with mail from this mailing list it will probably average 1-2 messages a month or less!

Discussion:

If you want to 'chat' about DNEWS with other DNEWS users discuss possible new features complain or praise the wonderful support we give or generally natter to other people running similar systems then you can join the DNEWS discussion list which is kindly hosted by Stephen White:

To subscribe send a message to:

dnews-request@adam.com.au

Containing one line:

subscribe

To unsubscribe send a message to

dnews-request@adam.com.au

 

Containing one line:

unsubscribe

 

To send messages to the list send to:

dnews@adam.com.au

 

[return to index]


Dnews Updates List

4.0j (5-Mar-97)

Installation Note for version 4.0:

With version 4 you need to allow for the xover index which could be upto 10% of your spool area by default these files will be placed in the WORKAREA directory but you can move them by defining XOVER_DIR in dnews.conf

4.0f (21-feb-97)

4.0a (5-Feb-97)

Some history on the reason for DNEWS 4.0

The DNEWS database was originally designed for efficiently sucking news items. And it was very good at this however for various reasonsabout 50% of DNEWS users take IHAVE feeds and although this worked well enough when running tests on Version 2.7 on very large news sites we found DNEWS was starting to struggle. With the projected growth of news being expected to double news rates every 1-2 years we took immediate action to improve the speed of the database.

Further investigation showed performance varied from .1 to 15 articles per second depending on type type of feed (ihave or suck) and the type of operating system (NT Unix VMS ...)

It became clear that any system based on storing articles in directories one directory for each newsgroup would not be able to handle the loads over the next 2 years.

As a result we decided to completely redesign the database. The new database would no longer use a group based directory structure. The design goals were:

DNEWS 4.0 is the result of this new database it succeeds in all the areas mentioned above and more besides typically it can take test articles at over 30 articles/second while still 50% idle. This makes DNEWS significantly faster than any other news server and probably the only technology currently available that will be able to take the increasing news loads over the next 2 years.

DNEWS 4 Technical Notes:

Unix Upgrade Notes (with multiple drives connected using symbolic links)

In dnews.conf use the setting:

spool_free true

And define each of your drives as one of the linked drives. e.g. if you currently have a second drive mounted as /var/spool/dnews/alt/binaries then your dnews.conf should look like this:

spool /var/spool/dnews
spool2 /var/spool/dnews/alt/binaries
diskspace 2000
diskspace2 1000

After the conversion has finished it is safe to move the directories but you cannot do this before the conversion because dnews must be able to find the 'old' files where they 'used to be'.

3.0j (11-12-96)

3.0h (11-12-96)

3.0g (10-12-96)

3.0f (9-12-96)

3.0e (7-12-96)

3.0d

3.0a

2.7q

2.7p

2.7o (12-Nov-96)

2.7n

2.7k

2.7j

2.7i

2.7h

2.7g

2.7f

2.7e

2.7d

2.7c

2.7b

2.7a

2.6q

2.6o

2.6n

2.6m

2.6k

2.6j

2.6i

2.6h

2.6g

2.6f

2.6e

2.6d (28-Aug-1996)

2.6c (27-Aug-1996)

2.6b (26-Aug-1996)

suck_stream true

This can triple the sucking speed or better on some systems.

2.6a (Beta release 25-Aug-96)

Summary of major new featres since 2.5i

2.5l (20-August-1996)

2.5k

2.5j

2.5i

2.5h

2.5g

2.5f

2.5e

2.5d

2.5c

2.5b

2.5a

2.4q

2.4p

2.4o

2.4n

2.4m (I don't like the letter l)

2.4k (Beta)

2.4j (Beta 19/June/1996)

2.4i (Beta 18/June/1996)

2.4h (Beta 17/June/1996)

2.4g (Beta 16/June/1996)

xover
xhdr
list
article
body

2.4f (Beta 6/June/1996)

2.4e (5/June/1996)

2.4d (19-5-96)

2.4c

2.4b

2.4a

nntp_feeder main.feeder.site *
!microsoft.*
nntp_suck2 alternate.feeder !*
alt.*
!microsoft.*
nntp_suck3 msnews.microsoft.com microsoft.*
update_at *5 * # Get main groups every 10 minutes
update_at2 20 3 # Get missing articles once a day
update_at3 40 9
12 # Get Micrsoft articles at 9:40 and 12:40
site me
groups *
site $nntp_feeder
groups *
!microsoft.*
builtin
posting
site msnews.microsoft.com
groups !*
microsoft.*
builtin
posting
...

Please don't update every 10 minutes from microsoft as they could easily get over loaded :-)

2.3p

replicate true
postonly true
site $NNTP_FEEDER
groups *
builtin
posting

2.3o (12/5/96)

2.3n (5/5/96)

2.3m (3/5/96)

2.3l (19/4/96)

2.3k

2.3j

2.3i

nntp_slow true
chan_nobuff true
tellnews stats_in
Shows stats in incoming feeds
tellnews stats_groups
Shows stats on group usage/size

2.3h

2.3g

2.3f

2.3e (21/Feb/96)

2.3d (11/Feb/96)

2.3c (7/Feb/96)

2.3b

2.3a

2.2g

2.2f (30/1/96)

2.2e

2.2d (20-Jan-1996)

2.2c (19-Jan-1996)

2.2a (15-Jan-1996)

2.1v (13-Jan-1996)

2.1u (12-Jan-1996)

2.1t (5-Jan-1996)

2.1s (3-Jan-1996)

2.1r

2.1q

2.1p

2.1o

2.1m (3/12/95)

2.1k

2.1j

2.1i (20/11/95)

2.1h

2.1g

2.1f (internal release mainly)

2.1e (4/11/1995)

2.1d (4/11/1995)

2.1c (2/11/1995)

2.1b (1/11/1995)

2.1a (31/10/1995)

2.0 Beta-5

2.0 Beta-4

2.0 Beta-3 (26-Oct-1995)

2.0 Beta-2 (24-Oct-1995)

2.0 Beta-1 (22-Oct-1995)

1.1g (30-Sep-1995)

tellnews purgelost
tellnews setused nnn

1.1f (27-Sep-1995)

1.1e (26-Sep-1995)

1.1d (20-Sep-1995)

1.1c (14-Sep-1995)

1.1b (Internal only)

1.1a (6-Sep-1995)

1.0e (Internal version)

1.0d (Internal version)

1.0c (22-Aug-1995)

1.0b (20-Aug-1995)

1.0a (14-Aug-1995)

check <messageID>
takethis <messageid>
mode stream

0.9d (Wide BETA release 2)

0.9c (Wide BETA release)

0.9b

Usage information for utilities

xmit

The xmit service performs serveral functions it can send a news feed to another site it can send news messages to a mail address (or email list) and it can build the full text searching database.

xmit site.name [-at minutes hours] [-mail list@address -name list-name]

site.nameThe site exactly as specified in newsfeeds.conf that this process should send items to
-at mins hoursSpecifies how often to wake up and attempt to send more articles
-mailSpecifies the email address to post articles (this is the gateway from news to email
-nameSpecifies the list name to insert in the headers of emailed messages.
-ftsSpecifies this feed should be used to build the fts search database

Examples:

Unix mail gateway
./xmit send.to.site.name -at "*2" "*" -mail JUGLIST@HERE.COM -name juggling"
NT news feed  (superseeded by live links)
addsvc -add dxmit1 xmit.exe "dxmit1 site.name.edu -at *5 *"
NT Full Text Searching
addsvc -add dfts xmit.exe "dfts ftsfeed -fts -at *5 *"

dposter

The dposter service is intended to only be used in conjunction with the drobot email robot. drobot writes email messages into a directory and dposter scans the directory periodically and then posts the email messages into the news server.

dposter -dir workarea -to news.server.name

-dir workareaThe directory to scan for messages written by drobot
-to news.serverThe name of the news server to post these messages into

Example:

   addsvc -add dposter dposter.exe "dposter -dir c:\ntmail\news
-to news.here.com"

drobot

drobot -dir workarea -groups group.name

This program should be run by NT-mail as a robot it will take the incoming message reformat some of the headers and write them to a directory ready for dposter to read.

Example:

   drobot -dir c:\ntmail\news -groups rec.juggling

addsvc

This utility is used for adding or removing services from the NT services list.

addsvc [-add servic_name program.exe "Name in service list"]
addsvc [-del servic_name]

-addAdd a service to the NT services list
-delRemove a service
service_nameA unique name for the service you can use this to delete the service later on
program.exeThe executable program to install typically it must be in the system32 directory
"parameters"This string is shown in the service list it is also used to emulate the parameters that would be used if the program was run from the command line.

Example:

   addsvc -add dfts xmit.exe "dfts ftsfeed -fts -at *5 *"

dtail

dtail -lines filename

This command can be used to show the last 'n' lines of a file it is a simplified version of the unix tail command. This is very useful for checking the end of the dnews.log file.

Example:

        dtail -30 dnews.log
[return to index] 

Managing Dnews

Renaming a Unix Volume

You may wish to rename a current volume /var/spool/news to be /var/spool/dnews to do this:

cd /var/spool/news
mkdir OLD
mv * OLD
umount /var/spool/news
emacs /etc/vfstab (or /etc/fstab)
(change /news into /dnews)
mount /var/spool/dnews
(When you are happy with the new system or you need the space:)
rm -r /var/spool/dnews/OLD

Posting to a moderated group

Note: You can only do this to your own moderated local groups.

Get the FreeAgent news reader. In agent.ini change showallfields to 1 e.g.

[Message]
ShowAllFields=1

Then re-post the article after putting your mail address in the approved header using freeagent.
HINT: when reading the log files it is easy to confuse an error message from DNEWS with an error message that dnews got while talking to your feeding site. Usually if the error looks like:

Status {223 transfer permission denied}

then it came from somewhere else not your system.
[return to index]

Creating local news groups

If you wish to have local groups on your server that are not on the server you 'suck' from then you should remove the 'POSTING' flag from the feed to your feeder site (in newsfeeds.conf). But only if you can get ihave or transfer permission to send articles up stream rather than simple 'posting' access.
You can add groups with the TELLNEWS NEWGROUP command e.g.

tellnews newgroup here.chat y myname Local group for chatting

here.chat = name of group
y = Normal group (m = moderated)
myname = Just a record of who created the group (one word)
Local group for chatting = description for active.names file.

To stop your local group from being sent up stream you would change the line in your newsfeeds.conf file:

groups * !here.chat !here.secret.*

NOTE: KNOWN BUG with version 2.0 items will still be sent upstream if you have left the 'POSTING' keyword in your feed. This will probably be fixed in a future version

Remote Control

Dnews on any platform may be controlled remotely the local machine may be running a different OS to the remote dnews machine hence it is possible to use the graphical interfaces to control non graphical versions of dnews. To set this up the local machine must have a copy of the tellnews executable and the tellnews.pass file the tellnews.pass file is a protected file on the dnews machine used to verify that the correct machine is sending commands.

Copy the dnews.conf the tellnews.pass and the tellnews executable to the local machine if you are using a different platform to the one that you are running dnews on then you'll have to download the tellnews executable. If you want to run the Windows GUI you'll need a copy of the dnewsman.exe file also. Edit the dnews.conf file and set up the workarea directory as the directory on the local machine where the tellnews.pass is. The dnews.conf file the dnewsman.exe and the tellnews.exe must be in the /winnt/system32 directory on Windows NT the /etc directory on Unix. The myname in the dnews.conf file must be set to the name or number of the remote machine. On the dnews server the machine that is to be used for remote control must have a read and post entry in the access.conf file.

In the dnews.conf on the local machine

myname remote.machine.name (this will be correct if you copy the file from the dnews machine)
workarea d:/dnews/spool (the name of the directory where the tellnews.pass is)

In the access.conf on the remote machine

remote.machine.name:Read
Post:::* (this is the name of the machine to send commands from)

Putting News On More Than One Drive

Unix: Symbolic links can be used to put specific hierachies on specific drives there is a pitfall associated with doing this however the percentage use on each of the drives in use needs to balanced to make the expire process work correctly. Alternatively the expire can be set up to apply a single rule to each hierachy that is on a different disk.

Windows NT: In the disk manager which is one of the control panels volume sets can be created and expanded these settings can be used to add one disk to another. The space that is being added must be empty and not allocated to another drive.
[return to index]

Giving DNEWS access to an INN news system.

Give normal NEWS READER access to the dnews machine by adding a line to your nnrp.access file
dnews.ip.name:read post:::*
Give DNEWS ihave access to your server Add a line to hosts.nntp
dnews.ip.name

[return to index]